JP2023070316A - Electric home appliance, portable information terminal, electric home appliance support system, and electric home appliance support method - Google Patents

Abstract

To provide an electric home appliance, a portable information terminal, an electric home appliance support system, and an electric home appliance support method that use a camera function of the portable information terminal to reflect on operation of the electric home appliance.SOLUTION: An electric home appliance support system primarily includes an image processing unit provided on a network, and is configured to be able to communicate with wireless LAN routers and portable information terminals in respective households via the network. The image processing unit includes: an analysis function that obtains captured images obtained by photographing a state of control target objects of electric home appliances with the portable information terminals and analyzes the state of the control target objects; and a control content determination function that determines control contents of the control target objects according to analysis results. The image processing unit transmits the analysis results and control contents to the wireless LAN routers and portable information terminals in the respective households via the network.SELECTED DRAWING: Figure 1

Description

The present invention relates to a home appliance, a mobile information terminal, a home appliance support system, and a home appliance support method, in which the home appliance communicates via a mobile information terminal.

Home appliances in recent years are equipped with a wide variety of functions due to the introduction and development of various technologies such as microcomputers, and more convenient lifestyles have been realized.

For example, according to Patent Document 1, regarding a heating cooker such as an IH or a range as a home appliance, "By controlling the heating means according to the state of the object to be cooked, the user can perform additional operations during or after cooking. Aiming to obtain a heating cooker capable of properly cooking an object to be cooked without any need, a heating means for heating an object to be cooked in a cooking chamber, an imaging means for photographing the object to be cooked, and the above-mentioned temperature detection means for detecting the temperature of the food to be cooked; and a control device for controlling the heating means based on the image data obtained from the photographing means and the temperature data obtained from the temperature detection means. is suggesting.

JP 2016-183785 A

According to Patent Document 1, the initial purpose can be achieved, and a highly convenient high-performance heating cooker has been realized. As a result, the product price becomes expensive and it is inevitable that the size increases. Therefore, if an equivalent function can be realized without mounting the camera function on the heating cooker, it is possible to provide an inexpensive and highly functional cooking cooker.

On the other hand, many home electric appliances other than cooking appliances generally do not have a camera function. Therefore, if a camera function is added to many home appliances other than these heating cookers, the function of the home appliances can be improved. I focused on the point.

In view of the above, it is an object of the present invention to provide a home appliance, a mobile information terminal, a home appliance support system, and a home appliance support method that utilize the camera function of the mobile information terminal to reflect the operation of the home appliance. .

From the above, in the present invention, "the main configuration is an image processing unit provided on the network, and is configured to be able to communicate with each home's wireless LAN router and mobile information terminal via the network, and the image processing unit , an analysis function that obtains an image of the state of the controlled object of the home appliance captured by the portable information terminal, analyzes the state of the controlled object, and control contents that determine the control contents of the controlled object according to the analysis result. determination function, and transmits analysis results and control contents to wireless LAN routers and personal digital assistants in each home via a network."

In addition, in the present invention, "a home appliance that obtains the control details determined by the home appliance support system via a wireless LAN router, and receives the control details obtained as a result of image analysis of the state of the control object of the home appliance" and a control function for setting the received control content and starting operation according to a start instruction.”

Further, in the present invention, "a portable information terminal for providing a photographed image of a state of an object to be controlled of a home appliance to a home appliance support system, wherein the portable information terminal is equipped with a home appliance application and displays the image. The screen is characterized by displaying a screen that prompts you to acquire an image of the state of the control object according to the instructions of the home appliance app, and a screen that displays the state of the control object and its analysis results and prompts you to start operating the home appliance. A mobile information terminal that

In addition, in the present invention, "the state of the control object of the home appliance is photographed by a portable information terminal, the state of the control object is analyzed with respect to the photographed image, and the control details of the home appliance are determined and controlled according to the analysis result. A home appliance support method characterized in that the content is transmitted to the home appliance, and the home appliance operates according to the control content.

Users can use highly functional products without installing many sensors, cameras, etc. on the home appliance side.

The figure which shows the structural example of the household appliance support system which concerns on Example 1 of this invention. The figure which shows the state of a controlled object when a household appliance is a heating cooker. The figure which shows the state of a controlled object when a household appliance is a washing machine. The figure which shows the state of a controlled object when a household appliance is a robot cleaner. The table which shows the correspondence of the analysis result of a picked-up image, and the control content of a heating cooker. A table showing the correspondence between the analysis result of the photographed image and the control details of the washing machine. A table showing the correspondence between the analysis result of the captured image and the control content of the robot cleaner. The figure which shows the processing flow in a heating cooker, an image processing part, and a heating cooker. The figure which shows the whole system configuration example which can support by giving the camera function of a portable information terminal to the typical household electrical appliances in general households.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

In the present invention, as a camera function that is not installed in home electric appliances, attention is paid to the camera function that a mobile information terminal has. Using the camera function of the mobile information terminal, the state of the control target of the home appliance is photographed, the photographed image is analyzed by the home appliance support system on the cloud side, the control contents based on the analysis result are transmitted to the home appliance, and this is transmitted to the home appliance. Control.

For this reason, in the following description, the first embodiment describes a home appliance support system on the cloud side, the second embodiment describes a cooking device not equipped with a camera function, and the third embodiment describes a general appliance not equipped with a camera function. In the fourth embodiment, a mobile information terminal will be described.

FIG. 1 shows a configuration example of a home appliance support system according to Embodiment 1 of the present invention. The home electric appliance support system 1 mainly includes an image processing unit 40, is provided on a network 30, and is configured to be able to communicate with a personal digital assistant 10 via a wireless LAN router 20 in each home. The portable information terminal 10 may be connected to the image processing section 40 without going through the wireless LAN router 20 .

In this figure, a personal digital assistant 10 is equipped with a home appliance application 13, and a photographed image of the state of an object controlled by a home appliance 60 is photographed by a camera function 11 in accordance with instructions from the home appliance application 13. P is transmitted from the terminal communication unit 12 to the wireless LAN router 20 in each home by wi-fi communication, and then transmitted to the image processing unit 40 via the network 30 .

The image processing unit 40, which is a main component of the home appliance support system 1, analyzes the state of the control target object of the home appliance 60 using the photographed image P that captures the state of the control target object. , the control content C of the home appliance 60 is determined based on the analysis result, and transmitted to the wireless LAN router 20 and the mobile information terminal 10 of each home via the network 30 . It is assumed that the image processing unit 40 is provided with an analysis result/control content reference table 50 shown in FIGS. 3a, 3b, and 3c.

After that, from the wireless LAN router 20 or the portable information terminal 10 in each home, the control content C is transmitted to the home appliance by wi-fi communication or Bluetooth communication, and the home appliance 60 follows the instructed control content C. carry out driving.

For this reason, the home appliance 60 has a communication function 61 by wi-fi or bluetooth, and a control function 62 in the home appliance 60 changes various operational settings according to the control content C instructed via transmission from the outside. It has the function of driving by

2a, 2b, and 2c are diagrams showing the state of the controlled object of the home appliance 60 photographed by the camera function 11. When the home appliance 60 is a heating cooker such as an IH or a microwave oven, the state shown in FIG. As shown, the object to be controlled is the foodstuff 20 (20a, 20b, 20c), and the state is the cooking condition (raw material, baking condition, etc.). When the home appliance 60 is a washing machine, the object to be controlled is the clothes 21 (21a, 21b, 21c) as shown in FIG. 2b. In the case of a machine, the object to be controlled is the floor 22 (22a, 22b, 22c) as shown in FIG.

3a, 3b, and 3c are analysis result/control content reference tables showing the correspondence relationship between the analysis result of the photographed image P in the image processing unit 40 and the control content C of the home appliance 60, and are created in advance and image processing is performed. It is held in the portion 40 . This analysis result/control content reference table is formed by image analysis and learning. Details of these figures will be described in detail in the following examples.

Here, the control content C of the home appliance 60 is the heating temperature and heating time in the case of a heating cooker, the washing time and the intensity of water flow in the case of a washing machine, and the robot cleaner in the case of a robot cleaner. Intensive cleaning of places where there is a lot of dust, etc. These can be appropriately set for each home electric appliance, and it is sufficient if these control contents C can be variably set. In the examples of FIGS. 3a, 3b, and 3c, the control content C is realized by designating an operation course or an operation menu.

It should be noted that the image processing unit 40 according to the first embodiment preferably prepares an analysis result/control details reference table for each type of home appliance, including the product type and model information from the home appliance application. By selecting an appropriate analysis result/control content reference table when image information is received, it is possible to provide optimum control content.

On the Internet, there are various types of product information with product type and model information as keys, and it is possible to search for this information and reflect it in the analysis result/control details reference table. Furthermore, it is possible to improve the accuracy by obtaining the image state of the control object again after the home appliance is operated by the control content C, analyzing it, and reflecting it in the analysis result / control content reference table. is.

According to the home appliance support system of the first embodiment, it is possible to provide optimum home appliance control contents from image information, and as a result, it is possible to realize home appliances with low cost and high functionality.

In a second embodiment, a case where the household appliance 60 is a heating cooker will be described. In this case, the heating cooker 60 (IH or microwave oven) is not equipped with a camera function, but as shown in FIG. It is assumed that the control function 62 has a function of changing various operational settings according to the control contents C instructed from the outside via transmission.

As shown in FIG. 1, the mobile information terminal 10 is equipped with a home appliance application 13 (heating cooker application). is photographed by the camera function 11 attached to the portable information terminal 10, and the photographed image is transmitted to the cloud side.

Here, the scene of application to heat cooking by the heat cooker application 13 includes before the start of operation and after operation, and the present invention can be applied to any of them. FIG. 4 shows the processing flow of the heating cooker application 13, divided into an initial setting stage, a pre-operation stage, and a post-operation stage. Further, FIG. 4 also shows an image processing unit side processing flow and a heating cooker side processing flow linked to the processing of the heating cooker application 13 .

Among these steps, the heating cooker application 13 is downloaded to the portable information terminal 10 in processing step S1 of the initial setting stage, and product information such as product and model is set in processing step S2.

After such setting is completed, application to heat cooking becomes possible, and in the pre-operation stage of the application scene, an instruction is given to photograph the foodstuff 20 to be cooked in the heat cooker in processing step S3, and the photographing is awaited. In the example of the ingredients 20 shown in FIG. 2a, 20a is fish, 20b is meat, and 20c is paella ingredients. It is transmitted to the home wireless LAN router 20 .

The image processing unit 40 that has received the captured image P performs image analysis of the captured image in processing step S11. According to this analysis, as illustrated in FIG. 3a, if the food in FIG. 2a is fish 20a, it turns out to be 20 cm frozen saury, and as illustrated in FIG. 3a, the food in FIG. 2a is paella. In the case of 20c, it is found that the ingredients are shrimp and rice, some of which are frozen and others are at room temperature. Note that the analysis method in this case can be realized by learning or other means, and a detailed explanation is omitted here.

Subsequently, in the image processing unit 40, in processing step S12, the content of control is determined from the analysis result. This determination can be realized by referring to the analysis result/control details reference table 50 created in advance. For example, when the food material in FIG. Depending on the result, as shown in FIG. 3a as an example of control contents C, it is determined that execution of the thawing and grilling course is necessary, and when the food material in FIG. Determine that execution is necessary. Also, the determination result is transmitted to the wireless LAN router 20 and the portable information terminal 10 of each home via the network 30 .

In response to this reception, the portable information terminal 10 displays the analysis results and the control details C in processing step S5, and prompts the user to operate the cooking start button of the heating cooker after confirmation in processing step S6.

Also, when this is received, in the heating cooker, in processing step S21, setting according to the control content C received by the receiving function 61 is performed by the control function 62, and it is determined that the user has operated the cooking start button in processing step S22. In response to the confirmation, in processing step S23, cooking is started according to the instructed control contents C. The above is the flow of the processing of each part in the pre-operation stage.

On the other hand, the processing flow in the post-operation stage among the application situations is basically the same as the above-mentioned pre-operation stage, so only different parts will be explained. The first point of difference is that when the user interrupts to check the cooking finish of the ingredients or completes the scheduled cooking course, the heating cooker determines and reports the end, which is detected in processing step S24. It is to be. Furthermore, the second point is that the middle stage or the end is displayed on the portable information terminal 10 in the processing step S7.

The third point is that, upon receiving this notification, the user takes another picture of the food after cooking in order to perform cooking again or additional cooking. The subsequent flow is basically the same as the processing of each part in the stage before the start of operation, but since it is after cooking, for example, the 15 cm sirloin of FIG. As a result, it is determined that additional heating for 10 minutes is required as control content C. Although the control content C in the pre-operation stage is to give a cooking menu, the control content C in the post-operation stage may be to directly instruct the additional heating time.

As a further usage scene of the present invention, it is possible to publish the processing result on SNS. In this case, the food after cooking is posted on the SNS via the mobile information terminal 10 . A user A can post information on how the product is cooked well, and a user B who has the same product owned by the user A can see the posted information, recognize the posted information with an application, and perform the same cooking process. In SNS, product information tags can be automatically added with an application.

According to the heating cooker of Example 2, it is possible to achieve a function equivalent to that of the equipped heating cooker while the cooking cooker is not equipped with a camera function.

In the second embodiment, in the case of a cooking device having a camera function, it has been described that the camera function is not installed and the cost is reduced by replacing it with the camera function of the mobile information terminal 10 . On the other hand, in the third embodiment, the camera function of the mobile information terminal 10 is added to a home appliance that does not have a camera function.

FIG. 5 shows an example of the overall system configuration that enables support by adding the camera function of the mobile information terminal 10 to typical household electrical appliances. Here, a washing machine 60C and a cleaning robot 60D are exemplified as home electric appliances to which the present invention can be applied other than the heating cooker (IH60B or microwave oven 60A).

Even in the case of the washing machine 60C and the cleaning robot 60D, the basic processing flow is the same as in FIG. However, in the case of the washing machine 60C, the clothes before washing are photographed as shown in FIG. The side analyzes and feeds back the result to the washing machine 60C to carry out washing.

Similarly, in the robot cleaner 60D, the state of the room before cleaning is photographed by the robot cleaner app as shown in FIG. It feeds back to the robot cleaner 60D, and drives the robot cleaner 60D based on the result.

In addition, in the washing machine analysis result/control content reference table 50 of FIG. 3B, for example, the detailed course menu is selected according to the analysis result that the work clothes are heavily soiled with oil but less muddy and have no wrinkles or ornaments. Designate as the control content, and according to the analysis results that there are no stains on the undergarments but are heavily soiled with mud, wrinkles and accessories, a menu with a careful course is designated as the control content, and coffee is spilled on the dress, According to the analysis result that there is no dirt, wrinkles, and ornaments, the menu for the fashionable course is designated as the control content.

In addition, in the analysis result/control content reference table 50 of the robot cleaner 60D in FIG. 3C, for example, according to the photographed photograph of the four corners of the living room on the floor, the dirt caused by dust is large, and although there is pet hair, the dirt caused by liquid and mud is small. According to the analysis result, we specified a menu with a careful course in the four corners as the control content, and according to the analysis result that there was no dust, hair, or sand according to the photograph of the kitchen floor, but there was milk spilled. The standard operation is specified as the content of control.

In the case of the robot cleaner 60D, if the robot cleaner 60D has learned the layout of the room to be cleaned, the layout of the furniture, etc., and has experience of the position information, it is possible to concentrate on cleaning the corner shown in the photograph.

The home appliances of the above-described second and third embodiments receive the control content obtained as a result of image analysis of the state of the control object by the receiving function 61, set the received control content in the control function 62, and start Operation is started according to instructions.

According to the third embodiment, it is possible to provide a new specification in which a camera function is substantially added to a home appliance that does not originally have a camera function.

Embodiment 4 describes a portable information terminal according to an embodiment of the present invention.

As described in the above embodiment, the mobile information terminal is equipped with a home appliance application, and its display screen displays a screen prompting the home appliance application to acquire an image of the state of the object to be controlled. A screen is displayed that displays the state of the object and its analysis results, and prompts the home appliance to start operating.

1: Support system 10: Personal digital assistant 11: Camera function 12: Terminal communication unit 13: Home appliance application 20: Wireless LAN router 30: Network 40: Image processing unit 50: Analysis result/control details reference table 60: Home appliance 60a : Microwave 60b: IH
60c: washing machine 60D: robot vacuum cleaner

Claims (6)

The main configuration is an image processing unit provided on the network, and it is configured to be able to communicate with each home's wireless LAN router and mobile information terminal via the network,
The image processing unit obtains a photographed image of the state of the control object of the home appliance photographed by the portable information terminal, and has an analysis function of analyzing the state of the control object, and the control object according to the analysis result. and a control content determination function for determining control content, and transmitting the analysis result and the control content to the wireless LAN router and the mobile information terminal in each home via the network. The home appliance support system of claim 1,
The home electric appliance support system, wherein the image processing unit holds a relationship between the analysis result and control details of the controlled object as a table, and refers to the table to determine the control details. A home appliance that obtains control details determined by the home appliance support system according to claim 1 or 2 via the wireless LAN router,
It is characterized by having a reception function for receiving control details obtained as a result of image analysis of the state of the control target of the home appliance, and a control function for setting the received control details and starting operation according to the start instruction. household appliances. The home appliance according to claim 3,
Home appliances are home appliances that are not equipped with a camera function, such as heating cookers, robot vacuum cleaners, or washing machines. A portable information terminal for providing the home appliance support system according to claim 1 or claim 2 with a photographed image of a state of an object to be controlled of the home appliance,
The portable information terminal is equipped with a home appliance application, and its display screen displays a screen prompting the user to obtain an image of the state of the controlled object according to instructions from the home appliance application, the state of the controlled object, and the analysis results thereof. A personal digital assistant, characterized in that a screen prompting the user to start driving is displayed. A state of an object to be controlled of a home appliance is photographed by a mobile information terminal, the state of the object to be controlled is analyzed with respect to the photographed image, control details of the home appliance are determined according to the analysis result, and the control content is determined by the home appliance. , and the home appliance operates according to the control contents.

Images