CN112629142A - Refrigerator virtual key operation processing method and refrigerator - Google Patents
Refrigerator virtual key operation processing method and refrigerator Download PDFInfo
- Publication number
- CN112629142A CN112629142A CN201910907530.7A CN201910907530A CN112629142A CN 112629142 A CN112629142 A CN 112629142A CN 201910907530 A CN201910907530 A CN 201910907530A CN 112629142 A CN112629142 A CN 112629142A
- Authority
- CN
- China
- Prior art keywords
- gesture
- user
- refrigerator
- virtual
- control panel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- F—MECHANICAL ENGINEERING; LIGHTING; HEATING; WEAPONS; BLASTING
- F25—REFRIGERATION OR COOLING; COMBINED HEATING AND REFRIGERATION SYSTEMS; HEAT PUMP SYSTEMS; MANUFACTURE OR STORAGE OF ICE; LIQUEFACTION SOLIDIFICATION OF GASES
- F25D—REFRIGERATORS; COLD ROOMS; ICE-BOXES; COOLING OR FREEZING APPARATUS NOT OTHERWISE PROVIDED FOR
- F25D29/00—Arrangement or mounting of control or safety devices
Landscapes
- Engineering & Computer Science (AREA)
- Chemical & Material Sciences (AREA)
- Combustion & Propulsion (AREA)
- Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Thermal Sciences (AREA)
- General Engineering & Computer Science (AREA)
- Cold Air Circulating Systems And Constructional Details In Refrigerators (AREA)
Abstract
The invention provides a refrigerator virtual key operation processing method and a refrigerator, wherein the method comprises the following steps: a projection step: after the refrigerator is detected to be opened, a virtual control panel is projected to a designated area of the storage compartment through the virtual projection module; an image acquisition step: acquiring gesture image information of a user in a designated area through an image acquisition module; a gesture recognition step: judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the gesture image information; a response step: if the gesture of the user is matched with the preset panel operation gesture, the operation of the user on the virtual control panel is detected and recognized, and an operation instruction corresponding to the operation is provided for the main control panel of the refrigerator, so that the main control panel can respond to the operation instruction. Based on the scheme of the invention, the operation identification efficiency can be improved, the energy consumption of the virtual projection module can be reduced, and the service life of the virtual projection module can be prolonged.
Description
Technical Field
The invention relates to the technical field of refrigerators, in particular to a refrigerator virtual key operation processing method and a refrigerator.
Background
At present, for the touch technology of refrigerator, set up touch panel or different operating button in order to control the running parameter etc. of refrigerator in the refrigerator door body or other box outsides mostly, mostly be the display panel display of carrying on the LED lamp, its simple structure and function singleness.
With the intellectualization of the refrigerator, a projection virtual key technology applied to the refrigerator is proposed, and the virtual operation key can increase the user-friendly experience. However, when the touch operation of the user on the virtual key is recognized, the error operation of the user often cannot be effectively distinguished, so that the user experience is reduced.
Disclosure of Invention
An object of the present invention is to recognize and respond to a user's effective operation gesture on a virtual control panel.
It is a further object of the present invention to effectively recognize a user's misoperation gesture on a virtual control panel.
Particularly, the present invention provides a method for processing a virtual key operation of a refrigerator, the refrigerator comprising: the box body defines a storage chamber; the virtual projection module and the image acquisition module are arranged in the storage room; and is
The method comprises the following steps:
a projection step: after the refrigerator is detected to be opened, a virtual control panel is projected to the designated area of the storage compartment through the virtual projection module;
an image acquisition step: acquiring gesture image information of a user in the designated area through the image acquisition module;
a gesture recognition step: judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the gesture image information;
a response step: if the gesture of the user is matched with the preset panel operation gesture, the operation of the user on the virtual control panel is detected and recognized, and an operation instruction corresponding to the operation is provided for the main control panel of the refrigerator, so that the main control panel can respond to the operation instruction.
Optionally, the gesture recognition step includes:
a characteristic extraction substep: extracting finger features in the gesture image information based on the gesture image information skin color segmentation processing;
an identifier sub-step: and judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the finger characteristics.
Optionally, the identifying sub-step comprises:
comparing the number of fingers of the user in the finger characteristics with a preset number; judging whether the gesture of the user is matched with a preset panel operation gesture or not;
if the number of the fingers of the user is larger than the preset number, judging that the gesture of the user is not matched with a preset panel operation gesture;
and if the number of the fingers of the user is not more than the preset number, judging that the gesture of the user is matched with a preset panel operation gesture.
Optionally, the feature extraction sub-step is further followed by:
a noise reduction processing substep: and performing expansion and corrosion treatment on the finger features.
Optionally, the gesture recognition step further comprises:
a pretreatment substep: preprocessing the gesture image information;
wherein the pre-treatment comprises at least one of: noise reduction processing and skin color segmentation processing.
Optionally, the gesture recognition step further includes:
an operation time acquisition step: if the gesture of the user is judged to be not matched with a preset panel operation gesture, acquiring the duration time of the mismatch of the gesture of the user and the preset panel operation gesture;
a first processing step: and if the duration is within the preset time, controlling the virtual projection module to continuously project the virtual control panel.
Optionally, the method further comprises:
a second processing step: and if the duration time exceeds the preset time, controlling the virtual projection module to enter a dormant state.
Optionally, the storage compartment comprises a refrigeration compartment;
the projecting step includes:
after the refrigerator is detected to be opened, a virtual control panel is projected to a designated area at the bottom of the refrigerating chamber through the virtual projection module, and an infrared signal is emitted to the designated area;
the designated area is located in a projection area at the bottom of the refrigerating chamber in a position-giving area of the refrigerating chamber, which is set for the door storage box.
Optionally, the responding step includes:
and if the gesture of the user is matched with a preset panel operation gesture, detecting and identifying the operation of the user on the virtual control panel based on the reflected signal of the infrared signal, and generating an operation instruction corresponding to the operation and providing the operation instruction to a main control panel of the refrigerator so as to enable the main control panel to respond to the operation instruction.
According to another aspect of the present invention, there is also provided a refrigerator including:
the box body defines a storage chamber;
the virtual projection module and the image acquisition module are arranged in the storage room;
the projection controller comprises a memory and a processor, wherein a program is stored in the memory, and the program is used for realizing the refrigerator virtual key operation processing method according to any one of the above items when being executed by the processor.
The invention provides a refrigerator virtual key operation processing method and a refrigerator. Based on the method provided by the invention, whether the operation of the user is misoperation can be judged before the operation of the user on the virtual control panel is identified, and whether further operation identification is carried out is further determined according to the judgment result; in addition, the scheme provided by the invention can also carry out sleep control on the virtual control panel after the misoperation lasts for a period of time, so that the energy consumption of the virtual projection module is reduced while the operation identification efficiency is improved, and the service life of the virtual projection module is prolonged.
The above and other objects, advantages and features of the present invention will become more apparent to those skilled in the art from the following detailed description of specific embodiments thereof, taken in conjunction with the accompanying drawings.
Drawings
Some specific embodiments of the invention will be described in detail hereinafter, by way of illustration and not limitation, with reference to the accompanying drawings. The same reference numbers in the drawings identify the same or similar elements or components. Those skilled in the art will appreciate that the drawings are not necessarily drawn to scale. In the drawings:
fig. 1 is a schematic view of a refrigerator according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a virtual projection module according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of relative positions of a designated area, a projection plane and a detection plane according to an embodiment of the invention;
FIG. 4 is a schematic view of the relative positions of the mounting box and the refrigeration compartment according to an embodiment of the invention;
FIG. 5 is a schematic view of a virtual projection module according to another embodiment of the present invention;
fig. 6 is a schematic view of a refrigerator according to another embodiment of the present invention;
fig. 7 is a flowchart illustrating a processing method for operating virtual keys of a refrigerator according to an embodiment of the present invention;
8a-c are schematic diagrams of finger features according to embodiments of the present invention;
fig. 9 is a flowchart illustrating a method for processing a virtual key operation of a refrigerator according to another embodiment of the present invention.
Detailed Description
Fig. 1 is a schematic structural diagram of a refrigerator 100 according to an embodiment of the present invention, and referring to fig. 1, the refrigerator 100 provided in this embodiment may include: a case 110 defining a storage compartment 111; the virtual projection module 120. As shown in fig. 2, the virtual projection module 120 may include: a projection assembly 121 configured to project a virtual control panel within a designated area of the storage compartment 111; a sensing component 122 configured to detect user operation in a designated area; and a processor 123 configured to recognize an operation of the virtual control panel by the user according to the detection result of the sensing assembly 122, and generate an operation instruction corresponding to the operation to be provided to the main control panel of the refrigerator 100 for the main control panel to respond to the operation instruction.
The refrigerator 100 that this embodiment provided, through utilizing projection subassembly 121 to project virtual control panel in the storing compartment 111 of refrigerator 100, be convenient for the inside function display of refrigerator 100 and control regulation, especially under the state that the refrigerator door body is in opening, can throw virtual control panel in order to make things convenient for the refrigerator 100 user to carry out different operations through the inside virtual projection module 120 that sets up of box 110 storing compartment 111 in the specified area, satisfy user's user demand, improve user experience. The processor 123 in this embodiment is mainly used as a core data processing component of the virtual projection module 120, and is in communication connection with the projection component 121 and the sensing component 122 to implement projection control of the projection component 121 and detection control of the sensing component 122, and determine whether the virtual control panel is operated by a user according to data detected by the sensing component 122, so as to send a corresponding operation instruction to the main control panel of the refrigerator 100. The virtual control panel projected by the projection component 121 may be an operation control panel for selecting relevant information such as an operation mode and an operation temperature of the refrigerator to put in the internal operation parameters of the refrigerator, and may also be an information display panel for presenting information such as a designated image, a picture, and article storage knowledge, and the virtual control panel projected by the projection component 121 may further have a plurality of touch keys to set the regulation and control requirements and the display requirements of different operation scenes of the refrigerator.
In the embodiment of the present invention, referring to fig. 1, the storage compartment 111 may include a refrigerating compartment 112; the projection assembly 121 and the sensing assembly 122 are disposed at the top of the refrigerating compartment 112. As can be seen from fig. 1 and 2, the virtual projection module 120 may further include a mounting box 124, and the projection element 121, the sensing element 122 and the processor 123 of the virtual projection module 120 may be disposed in the mounting box 124. The mounting box 124 may be provided with a transparent box cover, and the projection component 121 and the sensing component 122 are arranged in the mounting box 124 in parallel and located on one side of the mounting box 124 close to the refrigerator door. The mounting box 124 may be disposed at a middle position of the top of the refrigerating compartment 112 and at a distance within a predetermined distance range from the front end of the box body 110. In this embodiment, each component in the virtual projection module 120 can be integrated together through the mounting box 124, the position of each component is reasonably arranged, and the space occupied by the virtual projection module 120 inside the refrigerator 100 can be greatly reduced. In this embodiment, the projection module 121 is disposed on the top of the refrigerating compartment 112, so that the projection module 121 projects a virtual control panel vertically. The distance between the mounting box 124 and the front end of the box 110 is within a preset distance range, which can be further adjusted according to the size of the refrigerator 100, so that the projection distance is shortest, the projection angle is best, and the shielding effect is small.
For example, assuming that the length of the refrigerating compartment 112 of the refrigerator 100 is L (cm) and the width is W (cm), the center of the mounting box 124 is located at L/2, which provides a better projection position, and the projection shape is not deformed, considering the influence of the spacer rack objects on the projection, the distance M (cm) from the front end of the box 110 (specifically, the outer edge of the top of the refrigerator 100) is considered, and M < W/2, which is to prevent the bottle seat of the refrigerating compartment from colliding.
As can be seen from fig. 1 and 3, the designated area of the projection assembly 121 projecting the virtual control panel to the storage compartment 111 is located at the bottom of the refrigeration compartment 112; the projection assembly 121 and the sensing assembly 122 respectively form a projection surface and a detection surface in a designated area, and the projection surface and the detection surface are at least partially overlapped; the projection assembly 121 is further configured to project a virtual control panel in an overlapping area of the projection plane and the detection plane. Optionally, the designated area of the projection module 121 projecting the virtual control panel to the storage compartment 111 is preferentially located in the projection area of the refrigerator compartment 112 at the bottom of the refrigerator compartment 112, which is a relief area of the door storage box, so that the projection distance of the virtual projection module 120 is shortest, the projection angle is best, and the shielding effect is small.
In practical application, because the door body of the refrigerator is provided with the storage box for storing articles, the size of the storage plate arranged in the refrigerating chamber 112 is smaller than the internal size of the refrigerating chamber 112, and the refrigerating chamber 112 is provided with the yielding area arranged on the door body storage box, so that the door body storage box is arranged in the yielding area of the refrigerating chamber 112 after the door body of the refrigerator is closed and cannot touch the storage plate in the refrigerating chamber 112. In the solution provided in this embodiment, the virtual projection module 120 is disposed at the middle position of the top of the refrigerating compartment 112, and the designated area of the projection module 121 projecting the virtual control panel to the storage compartment 111 is preferentially located in the projection area of the refrigerating compartment 112 at the bottom of the refrigerating compartment 112 for the storage box of the door, so that not only the projection module 121 can completely project the virtual control panel at the bottom of the refrigerating compartment 112, but also the sensing module 122 can effectively detect the operation of the user in the designated area, thereby effectively reducing or even avoiding the shielding effect of the shelf in the refrigerating compartment 112 on the projection process of the virtual control panel or the process of detecting the operation of the user.
As shown in fig. 3, the designated area at the bottom of the refrigerating compartment 112 is area 1, the projection surface of the projection unit 121 is area 2, and the detection surface of the sensing unit 122 is area 3. Wherein both region 2 and region 3 belong to a part within region 1, and region 2 and region 3 have an overlapping region, region 4. The virtual control panel projected by the projection component 121 is located in the area 4. Assume that projection assembly 121 projects length L vertically1(cm) width W1(cm) virtual control panel, then, L1、W1Are all smaller than the letting area arranged on the storage box of the door body of the cold storage chamber 112The projected area at the bottom of the refrigeration compartment 112 has a maximum length and width.
As can be seen from fig. 2, in the embodiment of the present invention, the projection module 121 and the sensing module 122 are arranged in the mounting box 124 in parallel, and when the arrangement is specifically performed, the interval between the projection module 121 and the sensing module 122 can be controlled to be smaller than a certain distance, so that the projection plane of the projection module 121 and the detection plane of the sensing module 122 can be overlapped to the maximum extent, and the virtual control panel falls in the overlapping area of the projection plane and the detection plane, so that the operation of the virtual control panel by the user can be effectively detected through the sensing module 122 while the projection is implemented.
In an alternative embodiment of the present invention, the projection module 121 shown in fig. 2 may be configured to project a virtual control panel to a designated area and emit an infrared signal; and the sensing component 122 can be configured to detect the operation of the user in the designated area based on the infrared signal. Specifically, the projection module 121 in this embodiment may be a laser projection module composed of RGB three-color laser emitters, which emits imaging laser and projects a virtual control panel in a designated area. The virtual control panel may present information such as designated images and pictures, and may further include at least one virtual control key, based on which the operation parameters, the function modes, and the like of the refrigerator 100 may be set and selected. In this embodiment, the projection component may also emit an infrared signal during the process of projecting the virtual control panel to the designated area, and if the user touches the virtual control panel, the sensing component 122 may detect a reflected signal of the signal, so as to capture the related operations of the user in the designated area.
In another alternative embodiment of the present invention, the projection assembly 121 may also be configured to project a virtual control panel, which may be a laser projection module, to the designated area; the sensing component 122 may be configured to emit an infrared signal to the designated area and detect a user's operation within the designated area based on the infrared signal. In this embodiment, the sensing component 122 is used to send and receive infrared signals, and cooperate with the projection component 121 to detect the operation of the user in the designated area, so as to identify the operation of the user on the virtual control panel through the processor 123, and generate a corresponding operation instruction for the main control panel of the refrigerator 100 to respond. In this embodiment, the sensing component 122 is internally provided with an infrared emitting module and an infrared receiving module separately, so as to realize the emitting and receiving functions of the infrared signal.
As mentioned above, the virtual projection module 120 may be disposed on the top of the refrigerating compartment 112 of the box body 110, and preferably, the mounting box 124 may be vertically mounted on the top of the refrigerating compartment 112, such that the bottom surface of the mounting box 124 is disposed parallel to the top surface of the refrigerating compartment 112.
In an alternative embodiment of the present invention, as shown in fig. 4, a receiving portion 113 is provided at the front end of the top of the refrigerating compartment 112, and the mounting box 124 can be received in the receiving portion 113. Further, the bottom surface of the mounting box 124 and the top surface of the refrigerating compartment 112 are disposed in an inclined manner, and the inclined angle may be smaller than a certain included angle, and the included angle may be set according to actual requirements, which is not limited in the present invention. That is to say, in the present embodiment, a groove process may be adopted at the top of the refrigerating compartment 112 of the refrigerator 100, and the mounting box 124 may be disposed at the top of the refrigerating compartment 112 in a half-embedded manner, and is disposed obliquely with respect to the top surface of the refrigerating compartment 112, so as to further reduce the occupied space of the mounting box 124 for the refrigerating compartment 112, and make the mounting box 124 more easily integrated in the refrigerating compartment 112.
Because the projection ratio (α) of the virtual projection module is the projection distance (λ)/the picture width (w), where the range of α is 1.5-1.9, if the height of the refrigerating chamber is Hcm, and the range of H is 70cm-90cm, the range of the width of the picture projected by the virtual projection module is 36cm-60cm, and the current refrigerator 100 can meet the requirement of the module for projecting the picture size.
In an optional embodiment of the present invention, the refrigerator 100 may further include a door sensing device (not shown in the drawings) configured to detect a door opening event of the refrigerator 100 and generate a door opening signal; and, the projection component 121 is further configured to start after acquiring the door opening signal, so as to project the virtual control panel into the designated area of the storage compartment 111. In this embodiment, the door sensing device is arranged, the door opening signal is generated only when the door opening of the refrigerator 100 is detected, so that the projection assembly 121 projects the virtual control panel when acquiring the door opening signal, and the projection of the virtual control panel to the designated area can be automatically stopped after the door of the refrigerator 100 is closed, so as to further reduce energy consumption and save energy.
As shown in fig. 5, the refrigerator 100 may further include an image capturing module 130, which may be disposed in the mounting box 124 to capture image information of the refrigerating compartment 112 and assist the virtual projection module 120 in projection. The image capturing module 130 may be a camera or other devices with image capturing function.
According to the refrigerator 100 provided by the embodiment of the invention, the projection assembly 121 is used for projecting the virtual control panel in the storage compartment 111 of the refrigerator 100, so that the internal function display and control adjustment of the refrigerator 100 are facilitated, and particularly, when a refrigerator door is in an opened state, the virtual control panel can be projected in a designated area through the virtual projection module 120 arranged in the storage compartment 111 of the refrigerator body 110, so that different operations of a user of the refrigerator 100 are facilitated, the use requirements of the user are met, and the user experience is improved. Further, in the refrigerator 100 according to the embodiment of the present invention, the virtual projection module 120 is disposed in the middle of the top of the storage compartment 111 of the refrigerator body 110, so that the projection distance of the virtual projection module 120 is shortest, the projection angle is best, and the shielding effect is small. Moreover, by additionally arranging the door sensing device, the projection assembly 121 projects the virtual control panel when acquiring the door opening signal, and the projection of the virtual control panel to the designated area can be automatically stopped after the refrigerator 100 is closed, so that the energy consumption is further reduced, and the energy is saved.
In an alternative embodiment of the present invention, as shown in fig. 6, a projection controller 140 may be further disposed in the refrigerator 100, where the projection controller 140 may include a memory and a processor, and the memory stores a program, and the program is executed by the processor to implement the method for processing the virtual key operation of the refrigerator according to any one of the following embodiments of the present invention. The projection controller in this embodiment may be integrated with the processor 123 in the virtual projection module mentioned in the above embodiments, or may be a separately configured controller, which is not limited in the present invention.
Fig. 7 is a schematic flow chart of a processing method for operating virtual keys of a refrigerator according to an embodiment of the present invention, and as can be seen from fig. 7, the processing method for operating virtual keys of a refrigerator according to an embodiment of the present invention may include:
s702, a projection step: after the refrigerator is detected to be opened, the virtual projection module 120 projects a virtual control panel to the designated area of the storage compartment.
In the above embodiment, the door sensing device may be disposed in the refrigerator, so that, in step S702, the door sensing device may detect a door opening event of the refrigerator, and transmit information to the virtual projection module within a period of time (e.g., 2 seconds or other time) after the refrigerator is opened, so as to project the virtual control panel to the designated area of the storage compartment through the virtual projection module 120.
Optionally, the step S702 may further include: after the refrigerator is detected to be opened, a virtual control panel is projected to a designated area at the bottom of the refrigerating chamber through the virtual projection module 120, and an infrared signal is emitted to the designated area; the designated area is located in a projection area at the bottom of the refrigerating chamber 112 of a letting area of the refrigerating chamber, which is set for the door body storage box. In this embodiment, the virtual projection module 120 may be disposed at the middle position of the top of the refrigeration compartment, and the distance from the front end of the refrigeration compartment is within the preset distance range, so that the projection distance is shortest, the projection angle is best, and the shielding effect is small. In practical applications, the projection component in the virtual projection module 120 may emit an infrared signal while projecting the virtual control panel, or an infrared signal emitting device is separately disposed in the virtual projection module 120 to emit an infrared signal, which is not limited in the present invention.
In addition, the designated area of the projection component 121 projecting the virtual control panel to the storage compartment 111 is preferably located in the projection area of the refrigerating compartment 112 at the bottom of the refrigerating compartment 112, which is the escape area of the door storage box of the refrigerating compartment 112, so that not only can the projection component 121 completely project the virtual control panel at the bottom of the refrigerating compartment 112, but also the sensing component 122 can effectively detect the operation of a user in the designated area, and the shielding influence of the shelf in the refrigerating compartment 112 on the projection process of the virtual control panel or the operation process of the user can be effectively reduced or even avoided.
S704, an image acquisition step: the image capturing module 130 captures gesture image information of the user in the designated area.
After the virtual projection module 120 projects the virtual control panel to the designated area, the relevant operations of the virtual control panel can be identified. Optionally, in the embodiment of the present invention, before identifying the related operations of the virtual control panel, the image acquisition module 130 may first acquire gesture image information of a user in the designated area, and then quickly determine whether the operation of the user on the virtual control panel is a normal operation, a misoperation, or a blocking operation according to the acquired gesture image information, so as to improve the operation identification efficiency of the virtual projection module 120 on the user, and at the same time, avoid the influence on the normal operation of the refrigerator due to the response to the misoperation of the user. The image acquisition device can be a camera or other equipment with an image acquisition function.
S706, gesture recognition: and judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the gesture image information.
In this embodiment, after the gesture image information is acquired, the gesture image information may be preprocessed first, where the preprocessing includes at least one of the following: noise reduction processing and skin color segmentation processing. After the gesture image information is preprocessed, a feature extraction substep may be performed: extracting finger features in the gesture image information based on the gesture image information skin color segmentation processing; an identifier sub-step: and judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the finger characteristics. The method has great difference with the background in HSV (Hue, Saturation and brightness) color space, so that the finger features can be extracted from the gesture image information by utilizing the skin color features, and further the finger features can be effectively identified. Optionally, after the specified feature extraction sub-step, a noise reduction sub-step, i.e., a dilation and erosion process on the finger features, may also be performed. In the morphological processing of the image, the erosion is to use an algorithm to erode the edges of the image. The effect is to "flash" off the edges of the target. Dilation is the use of an algorithm to expand the edges of the image. The effect is to fill out the edge or inner pits of the target. The same number of erosion and swelling is used, so that the target surface can be smoother, and the influence of noise interference on gesture recognition is further reduced.
The finger feature may be the number of fingers of the user included in the gesture image information. When the gesture of the user is judged based on the gesture characteristics, the number of fingers of the user in the finger characteristics can be compared with the preset number; judging whether the gesture of the user is matched with a preset panel operation gesture or not; if the number of the fingers of the user is larger than the preset number, judging that the gesture of the user is not matched with the preset panel operation gesture; and if the number of the fingers of the user is not more than the preset number, judging that the gesture of the user is matched with the preset panel operation gesture.
For example, the preset number may be set to 2, when the number of fingers of the user is identified to be greater than or equal to 2 based on the gesture image information, the gesture of the user is considered to be not matched with the preset panel operation gesture, that is, the user is determined to be a misoperation currently, and the gesture of the user is considered to be matched with the preset panel operation gesture only when the number of fingers of the user is 1. The gesture matching the preset panel operation gesture may be as shown in fig. 8a, and the non-matching gesture may be as shown in fig. 8b and 8 c. In practical application, different preset panel operation gestures can be set according to different types of users, and the invention is not limited.
S708, the responding step: if the gesture of the user is matched with the preset panel operation gesture, the operation of the user on the virtual control panel is detected and recognized, and an operation instruction corresponding to the operation is provided for the main control panel of the refrigerator, so that the main control panel can respond to the operation instruction.
As mentioned above, in step S702, while the virtual control panel is projected by the virtual projection module, an infrared signal may be emitted to the designated area. Accordingly, in the step S708, if the gesture of the user matches the preset panel operation gesture, the operation of the user on the virtual control panel may be detected and recognized based on the reflected signal of the infrared signal, and an operation instruction corresponding to the operation is provided to the main control panel of the refrigerator, so that the main control panel responds to the operation instruction. For example, the intensity of the reflected signal of the infrared signal may be detected by an infrared sensor, and the operation of the virtual control panel by the user may be deduced. During the operation period of the user, the average filtering processing is performed on the infrared signal data to prevent misjudgment.
In the above embodiments, the gesture of the user is matched with the preset panel operation gesture. In practical applications, the gesture of the user may not match the preset panel operation gesture.
That is, after the step S706, the method may further include:
an operation time acquisition step: if the gesture of the user is judged not to be matched with the preset panel operation gesture, acquiring the duration time of the gesture of the user which is not matched with the preset panel operation gesture; a first processing step: if the duration is within the preset time, the virtual projection module 120 is controlled to continue projecting the virtual control panel. A second processing step: if the duration time exceeds the preset time, the virtual projector module 120 is controlled to enter the sleep state.
That is, when it is determined that the gesture of the user is not matched with the preset panel operation gesture based on the gesture image information captured by the image capturing module 130, it may be further determined that the gesture of the current user is a misoperation gesture or a blocking gesture (for example, it is determined whether the gesture of the user is a hand based on the gesture image information, if fig. 8c is a blocking gesture, and fig. 8b is a misoperation gesture), and the state maintaining time is within the preset time (if less than N seconds), the virtual projection module 120 performs the mute processing on the operation of the user, and continues to maintain the current projection state. If it is further determined that the gesture of the user maintains the misoperation gesture or the blocking gesture exceeds the preset time (greater than or equal to N seconds), the virtual projection module 120 may enter a sleep state, i.e., a low power consumption mode, and then the virtual projection module 120 is turned off after a period of time. According to the scheme provided by the embodiment, the gesture of the misoperation of the user is maintained for more than or equal to the preset time and is judged as the misoperation gesture, the gesture switching is prevented from being frequently started on the virtual projection module 120, the power consumption of the virtual projection module 120 is reduced, the service life of the virtual projection module is verified, and the user experience is further improved.
The following describes a method for processing a virtual key operation of a refrigerator according to an embodiment of the present invention in detail by using an alternative embodiment. As shown in fig. 9, a method for processing a virtual key operation of a refrigerator according to an embodiment of the present invention may include:
s902, detecting a door opening event of the refrigerator;
s904, starting the image acquisition module 130 and the virtual projection module 120;
s906, acquiring gesture image information of a user through the image acquisition module 130, and transmitting an infrared signal by using the virtual projection module 120;
s908, judging whether the current gesture of the user is matched with a preset panel operation finger or not based on the gesture image information; specifically, finger features in the gesture image information can be extracted, the number of fingers can be recognized, and whether the current gesture of the user is matched with a preset panel operation gesture or not is judged by comparing the number of fingers with 1; if yes, go to step S912; if not, go to step S910;
s910, determining that the user is not misoperation currently;
s912, acquiring a reflection signal of the infrared signal, and filtering the reflection signal;
s914, identifying the operation of the user on the virtual control panel;
s916, performing expansion and corrosion treatment on the identified finger characteristics; the influence of noise interference on gesture recognition is reduced;
s918, judging whether the current gesture of the user is a shielding gesture; when the shielding gesture is determined, whether the fingers of the user are all fingers can be determined, if the number of the collected fingers of the user is 5, the current gesture of the user is determined to be the shielding gesture, and step S920 is executed; if the gesture is smaller than 5, judging that the gesture is not a shielding gesture, namely the gesture is a misoperation, and executing a step S922;
s920, determining that a shielding object exists on the preset projection path of the virtual projection module 120, the virtual projection module 120 does not respond, and closing the virtual projection module 120 when the shielding object is continuously detected within a period of time; and proceeds to step S906;
s922, judging that the current gesture of the user is a misoperation gesture, and the virtual projection module 120 does not respond; and proceeds to step S906.
The embodiment of the invention provides a refrigerator virtual key operation processing method and a refrigerator, in the method provided by the embodiment of the invention, after the refrigerator is opened, gesture image information of a user in a specified area is acquired through an image acquisition module 130, whether the gesture of the user is matched with a preset panel operation gesture is judged, so that whether the user is misoperation is identified, and the operation of the user is identified and responded only under the condition that the gesture of the user is not misoperation. Based on the method provided by the embodiment of the invention, whether the operation of the user is misoperation can be judged before the operation of the user on the virtual control panel is identified, and whether further operation identification is carried out is further determined according to the judgment result; in addition, the scheme provided by the embodiment of the invention can also perform sleep control on the virtual control panel after the misoperation lasts for a period of time, so that the energy consumption of the virtual projection module 120 is reduced while the operation identification efficiency is improved, and the service life of the virtual projection module is prolonged.
Thus, it should be appreciated by those skilled in the art that while a number of exemplary embodiments of the invention have been illustrated and described in detail herein, many other variations or modifications consistent with the principles of the invention may be directly determined or derived from the disclosure of the present invention without departing from the spirit and scope of the invention. Accordingly, the scope of the invention should be understood and interpreted to cover all such other variations or modifications.
Claims (10)
1. A refrigerator virtual key operation processing method, the refrigerator comprising: the box body defines a storage chamber; the virtual projection module and the image acquisition module are arranged in the storage room; and is
The method comprises the following steps:
a projection step: after the refrigerator is detected to be opened, a virtual control panel is projected to the designated area of the storage compartment through the virtual projection module;
an image acquisition step: acquiring gesture image information of a user in the designated area through the image acquisition module;
a gesture recognition step: judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the gesture image information;
a response step: if the gesture of the user is matched with the preset panel operation gesture, the operation of the user on the virtual control panel is detected and recognized, and an operation instruction corresponding to the operation is provided for the main control panel of the refrigerator, so that the main control panel can respond to the operation instruction.
2. The method of claim 1, wherein the gesture recognition step comprises:
a characteristic extraction substep: extracting finger features in the gesture image information based on the gesture image information skin color segmentation processing;
an identifier sub-step: and judging whether the gesture of the user is matched with a preset panel operation gesture or not based on the finger characteristics.
3. The method of claim 2, wherein the identifying substep comprises:
comparing the number of fingers of the user in the finger characteristics with a preset number; judging whether the gesture of the user is matched with a preset panel operation gesture or not;
if the number of the fingers of the user is larger than the preset number, judging that the gesture of the user is not matched with a preset panel operation gesture;
and if the number of the fingers of the user is not more than the preset number, judging that the gesture of the user is matched with a preset panel operation gesture.
4. The method of claim 2, wherein the feature extraction substep is further followed by:
a noise reduction processing substep: and performing expansion and corrosion treatment on the finger features.
5. The method of claim 2, wherein the gesture recognition step further comprises:
a pretreatment substep: preprocessing the gesture image information;
wherein the pre-treatment comprises at least one of: noise reduction processing and skin color segmentation processing.
6. The method of claim 1, wherein the gesture recognition step is further followed by:
an operation time acquisition step: if the gesture of the user is judged to be not matched with a preset panel operation gesture, acquiring the duration time of the mismatch of the gesture of the user and the preset panel operation gesture;
a first processing step: and if the duration is within the preset time, controlling the virtual projection module to continuously project the virtual control panel.
7. The method of claim 6, further comprising:
a second processing step: and if the duration time exceeds the preset time, controlling the virtual projection module to enter a dormant state.
8. The method of claim 1, wherein the storage compartment comprises a refrigeration compartment;
the projecting step includes:
after the refrigerator is detected to be opened, a virtual control panel is projected to a designated area at the bottom of the refrigerating chamber through the virtual projection module, and an infrared signal is emitted to the designated area;
the designated area is located in a projection area at the bottom of the refrigerating chamber in a position-giving area of the refrigerating chamber, which is set for the door storage box.
9. The method of claim 8, wherein the responding step comprises:
and if the gesture of the user is matched with a preset panel operation gesture, detecting and identifying the operation of the user on the virtual control panel based on the reflected signal of the infrared signal, and generating an operation instruction corresponding to the operation and providing the operation instruction to a main control panel of the refrigerator so as to enable the main control panel to respond to the operation instruction.
10. A refrigerator, comprising:
the box body defines a storage chamber;
the virtual projection module and the image acquisition module are arranged in the storage room;
a projection controller comprising a memory and a processor, wherein the memory stores a program, and the program is used for realizing the refrigerator virtual key operation processing method according to any one of claims 1 to 9 when the program is executed by the processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910907530.7A CN112629142A (en) | 2019-09-24 | 2019-09-24 | Refrigerator virtual key operation processing method and refrigerator |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910907530.7A CN112629142A (en) | 2019-09-24 | 2019-09-24 | Refrigerator virtual key operation processing method and refrigerator |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112629142A true CN112629142A (en) | 2021-04-09 |
Family
ID=75282954
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910907530.7A Pending CN112629142A (en) | 2019-09-24 | 2019-09-24 | Refrigerator virtual key operation processing method and refrigerator |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112629142A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130030552A1 (en) * | 2011-07-28 | 2013-01-31 | Bryan James Beckley | Optically-projected user interface for appliances |
| US20160209928A1 (en) * | 2015-01-16 | 2016-07-21 | Samsung Electronics Co., Ltd. | Virtual input device and method for receiving user input using the same |
| CN107197223A (en) * | 2017-06-15 | 2017-09-22 | 北京有初科技有限公司 | The gestural control method of micro-projection device and projector equipment |
| CN108153428A (en) * | 2018-01-26 | 2018-06-12 | 哨鸟（深圳）前海科技有限公司 | Laser-projection keyboard and its implementation |
| WO2018197224A1 (en) * | 2017-04-28 | 2018-11-01 | BSH Hausgeräte GmbH | Display of information on a refrigeration appliance |
| CN208936629U (en) * | 2018-10-12 | 2019-06-04 | 合肥美的电冰箱有限公司 | Refrigerator with projection operation's function |
-
2019
- 2019-09-24 CN CN201910907530.7A patent/CN112629142A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20130030552A1 (en) * | 2011-07-28 | 2013-01-31 | Bryan James Beckley | Optically-projected user interface for appliances |
| US20160209928A1 (en) * | 2015-01-16 | 2016-07-21 | Samsung Electronics Co., Ltd. | Virtual input device and method for receiving user input using the same |
| WO2018197224A1 (en) * | 2017-04-28 | 2018-11-01 | BSH Hausgeräte GmbH | Display of information on a refrigeration appliance |
| CN107197223A (en) * | 2017-06-15 | 2017-09-22 | 北京有初科技有限公司 | The gestural control method of micro-projection device and projector equipment |
| CN108153428A (en) * | 2018-01-26 | 2018-06-12 | 哨鸟（深圳）前海科技有限公司 | Laser-projection keyboard and its implementation |
| CN208936629U (en) * | 2018-10-12 | 2019-06-04 | 合肥美的电冰箱有限公司 | Refrigerator with projection operation's function |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10402624B2 (en) | Presence sensing | |
| US9268412B2 (en) | Input apparatus having an input recognition unit and input recognition method by using the same | |
| KR101660215B1 (en) | Presence sensing | |
| US20190102597A1 (en) | Method and electronic device of performing fingerprint recognition | |
| US20160116891A1 (en) | Method and device for operating a household appliance and household appliance | |
| JP6671288B2 (en) | Gesture device, operation method thereof, and vehicle equipped with the same | |
| JP2005269891A (en) | Control and management of electric power of electronic equipment utilizing detection of eye | |
| JP5532315B2 (en) | Image processing apparatus, image processing method, program, and electronic apparatus | |
| CN106247561A (en) | A kind of air-conditioning and long-range control method thereof and device | |
| US11570381B2 (en) | Method and apparatus of adaptive infrared projection control | |
| WO2021212993A1 (en) | Camera module in refrigeration appliance and method for preventing lens from fogging | |
| CN105783413A (en) | Method of obtaining information of articles stored in refrigerator and refrigerator | |
| US11335010B2 (en) | Methods for viewing and tracking stored items | |
| CN108592514A (en) | Intelligent refrigerator and its interaction control method | |
| JP2011118533A (en) | Device and method for inputting touch position | |
| KR20210131891A (en) | Method for authentication or identification of an individual | |
| US9305207B2 (en) | Depth-photographing method of detecting human face or head | |
| CN211372855U (en) | Refrigerator with internal projection function | |
| CN113063261A (en) | Refrigerator control method, refrigerator and computer-readable storage medium | |
| CN113012656A (en) | Display screen control method, intelligent household appliance and storage medium | |
| CN112556267B (en) | Lighting control method for refrigerator compartment and refrigerator | |
| CN112629142A (en) | Refrigerator virtual key operation processing method and refrigerator | |
| CN112629105A (en) | Virtual control panel operation identification method and refrigerator | |
| CN211120191U (en) | Refrigerator with a door | |
| KR101853276B1 (en) | Method for detecting hand area from depth image and apparatus thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210409 |