CN108880957B - Household appliance and remote control method thereof, server and terminal control equipment - Google Patents

Household appliance and remote control method thereof, server and terminal control equipment Download PDF

Info

Publication number
CN108880957B
CN108880957B CN201810618496.7A CN201810618496A CN108880957B CN 108880957 B CN108880957 B CN 108880957B CN 201810618496 A CN201810618496 A CN 201810618496A CN 108880957 B CN108880957 B CN 108880957B
Authority
CN
China
Prior art keywords
time
household appliance
reservation
information
voice
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201810618496.7A
Other languages
Chinese (zh)
Other versions
CN108880957A (en
Inventor
彭碧海
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Midea Group Co Ltd
GD Midea Air Conditioning Equipment Co Ltd
Original Assignee
Midea Group Co Ltd
GD Midea Air Conditioning Equipment Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Midea Group Co Ltd, GD Midea Air Conditioning Equipment Co Ltd filed Critical Midea Group Co Ltd
Priority to CN201810618496.7A priority Critical patent/CN108880957B/en
Publication of CN108880957A publication Critical patent/CN108880957A/en
Application granted granted Critical
Publication of CN108880957B publication Critical patent/CN108880957B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L12/00Data switching networks
    • H04L12/28Data switching networks characterised by path configuration, e.g. LAN [Local Area Networks] or WAN [Wide Area Networks]
    • H04L12/2803Home automation networks
    • H04L12/2816Controlling appliance services of a home automation network by calling their functionalities
    • H04L12/282Controlling appliance services of a home automation network by calling their functionalities based on user interaction within the home
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/02Feature extraction for speech recognition; Selection of recognition unit
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/26Speech to text systems
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/28Constructional details of speech recognition systems
    • G10L15/30Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223Execution procedure of a spoken command

Landscapes

  • Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Signal Processing (AREA)
  • Telephonic Communication Services (AREA)

Abstract

The invention discloses a household appliance and a remote control method thereof, a server and a terminal control device, wherein the remote control method comprises the following steps: acquiring voice information; performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information; and when the reserved time is up, controlling the household appliance to operate according to the generated reserved instruction of the household appliance. The invention solves the technical problem of inaccurate control time when the household electrical appliance is controlled to operate in advance in the prior art, not only ensures convenient operation for users, but also realizes effective control of the advanced operation of the household electrical appliance.

Description

Household appliance and remote control method thereof, server and terminal control equipment
Technical Field
The invention relates to the field of household appliances, in particular to a household appliance, a remote control method thereof, a server and a terminal control device.
Background
With the intelligent development of household appliances, more and more household appliances realize an automatic control function and even a remote control function. For example, before the user returns home, the user sends a power-on command to the home appliance through the mobile terminal remotely to control the home appliance to be turned on in advance, so that the user can enjoy a comfortable environment when returning home.
However, when the user remotely controls the household appliance, the APP needs to be opened and operated in the APP, so that the operation is inconvenient. Moreover, the user cannot accurately estimate the starting time of the household appliance, so that energy is wasted due to too early starting, and the comfort of the user is affected due to too late starting, so that the control of the household appliance cannot be effectively realized.
Disclosure of Invention
The embodiment of the application provides the household appliance, the remote control method thereof, the remote server and the terminal control device, and solves the technical problem that the control time is inaccurate when the household appliance is controlled to run in the prior art.
The embodiment of the application provides a remote control method of household electrical appliance equipment, which comprises the following steps:
acquiring voice information;
performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result;
generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information;
and when the reserved time is up, controlling the household appliance to operate according to the generated reserved instruction of the household appliance.
Optionally, the voice information is collected by a terminal control device, and the terminal control device performs voice recognition on the collected voice information and extracts preset feature information from a voice recognition result.
Optionally, the voice information is collected by the terminal control device and sent to the voice server; and the voice information is subjected to voice recognition by the voice server, and preset characteristic information is extracted from a voice recognition result.
Optionally, the step of performing voice recognition on the acquired voice information and extracting preset feature information from a voice recognition result includes:
carrying out voice recognition on the acquired voice information to obtain character information corresponding to the voice information;
and carrying out language analysis on the character information to obtain preset characteristic information.
Optionally, the preset feature information includes departure time and a traffic mode.
Optionally, the step of generating a reservation instruction and a reservation time of the home appliance according to the extracted feature information includes:
estimating the distance time required to be spent between the departure place and the destination according to the transportation mode;
and acquiring the appointment time of the household appliance according to the starting time and the estimated distance time, and generating an appointment instruction of the household appliance.
Optionally, the step of determining a reservation time of the home appliance according to the departure time and the estimated journey time, and generating a reservation instruction of the home appliance includes:
acquiring target operation parameters of the household appliance;
estimating the running time of the household appliance reaching the target running parameter according to the target running parameter of the household appliance;
and acquiring the appointment time of the household appliance according to the starting time, the estimated distance time and the running time of the household appliance, and generating an appointment instruction of the household appliance.
Optionally, the preset feature information further includes a transaction processing time in the middle; the estimated journey time is added with the transaction processing time in the middle.
Optionally, before the step of controlling the operation of the home appliance device according to the generated reservation instruction of the home appliance device when the reservation time arrives, the method further includes:
acquiring the movement condition of terminal control equipment in real time;
and re-determining the reserved time of the household appliance according to the movement condition of the terminal control equipment.
Optionally, the movement condition of the terminal control device includes a geographic position and/or a movement speed of the terminal control device; the step of re-determining the reserved time of the household appliance according to the movement condition of the terminal control equipment comprises the following steps:
according to the acquired geographic position and/or movement speed of the terminal control equipment, the journey time required to reach the destination is estimated again;
and re-determining the reservation time of the household appliance according to the re-estimated distance time.
Optionally, before the step of controlling the operation of the home appliance device according to the generated reservation instruction of the home appliance device when the reservation time arrives, the method further includes:
if new voice information is acquired, performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; and re-determining the reserved time of the household appliance according to the extracted characteristic information.
Optionally, when voice information sent by a plurality of terminal control devices is acquired, performing voice recognition, extracting preset feature information, and generating corresponding reservation instructions and reservation time, respectively; and when the first arrived reservation time arrives, controlling the household appliance to operate according to the reservation instruction corresponding to the reservation time.
Optionally, the household appliance comprises an air conditioning device.
The embodiment of the present application further provides a server, which includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and when being executed by the processor, the computer program implements the steps of the remote control method for the home appliance of any one of the above-mentioned implementation methods, and controls the operation of the home appliance.
The embodiment of the application also provides terminal control equipment, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the computer program is used for realizing the acquisition of voice information when being executed by the processor.
Optionally, the computer program, when executed by the processor, further implements the steps of the remote control method for the home appliance according to any one of the above embodiments.
The embodiment of the application further provides a household appliance, which comprises a memory, a processor and a computer program, wherein the computer program is stored on the memory and can be run on the processor, the computer program is executed by the processor, receives the reservation instruction and the reservation time of the household appliance, and when the reservation time arrives, the household appliance is controlled to run according to the reservation instruction of the household appliance.
Optionally, the reservation instruction and the reservation time of the home appliance device are sent to the home appliance device by the terminal control device through the remote server.
The embodiment of the present application further provides a computer-readable storage medium, where a remote control application program is stored on the computer-readable storage medium, and the remote control application program is executed by a processor to implement the steps of the remote control method for a home appliance according to any one of the above embodiments, so as to control the home appliance to operate in advance.
One or more technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:
1. the method comprises the steps of extracting preset characteristic information from acquired voice information, generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information, and sending the reservation instruction of the household appliance to the household appliance when the reservation time is up to control the household appliance to operate. Therefore, the user only needs to send out voice information to realize the advanced operation control of the household appliance, and the operation of the user is very convenient.
2. The reserved time of the household appliance is determined according to the feature information extracted from the voice information and is not manually controlled by a user, so that the effective control of the advanced operation of the household appliance is realized.
3. In the embodiment of the invention, the terminal control equipment is used for voice recognition, thereby reducing the burden of the server; in addition, the terminal control equipment utilizes a strong voice recognition server to perform voice recognition, so that the accuracy of the voice recognition is improved, and the effective control of the household appliance is further improved.
Drawings
Fig. 1 is a diagram illustrating an exemplary system configuration for remote control of a home appliance according to an embodiment of the present invention;
FIG. 2 is a diagram of exemplary functional modules of a server according to an embodiment of the present invention;
fig. 3 is a diagram illustrating an exemplary hardware module of a terminal control device according to an embodiment of the present invention;
fig. 4 is a schematic flow chart of a first embodiment of a remote control method for a home appliance according to an embodiment of the present invention;
FIG. 5 is a schematic diagram of a detailed flow chart of an embodiment of step S2 in FIG. 4;
FIG. 6 is a schematic diagram illustrating a detailed flow of one embodiment of step S3 in FIG. 4;
FIG. 7 is a schematic diagram of a detailed flow chart of an embodiment of step S32 in FIG. 6;
fig. 8 is a flowchart illustrating a second embodiment of a remote control method for a home appliance according to an embodiment of the present invention.
Detailed Description
For a better understanding of the above technical solutions, exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The technical scheme of the invention mainly aims to solve the technical problems that the operation of a user is inconvenient and the advanced operation of the household appliance cannot be effectively controlled in the prior art, and provides a scheme for realizing the remote control of the household appliance. Therefore, the user can control the running of the household appliance only by sending out corresponding voice, so that the operation of the user is convenient, the running time of the household appliance is not manually controlled by the user any more, but the control time is estimated according to the voice information, and the advanced control of the household appliance can be effectively realized.
As shown in fig. 1, fig. 1 is a diagram illustrating a system configuration for remote control of a home appliance according to an embodiment of the present invention.
The remote control system of the home appliance includes a terminal control device 100, a server 200, a home appliance 300, and a controller 400. The terminal control device 100 can be carried by a user, and has a function of performing remote communication with the server 200, for example, including a mobile phone, a tablet computer, a smart wearable device, a voice device, and the like. The server 200 establishes communication with the terminal control device 100 and the home appliance 300, respectively, to realize remote control of the home appliance 300 by the terminal control device 100. The controller 400 may be disposed in the home device 300, and may even be the same as the control module of the home device 300, or may be disposed independently of the home device 300. The controller 400 and the server 200 establish communication, so that the controller 400 can control the operation of the home appliance 300 according to the control command sent by the server 200. The controller 400 may also receive an instruction sent by another device (e.g., a remote controller), and control the operation of the home appliance 300 according to the instruction.
The above-mentioned household electrical appliance 300 includes, for example and without limitation, an air conditioning device. The above mentioned terminal control device is, for example, but not limited to, a mobile phone, a tablet computer, a wearable device, and other terminal devices with a remote control function.
Specifically, in an embodiment, as shown in fig. 2, fig. 2 is a schematic functional module diagram of a server 200 according to an embodiment of the present invention. The server 200 includes a processor 201, a memory 202, and a first communication module 203. In an example, the server 200 may be a single server, or may be a server cluster composed of a plurality of server units.
It will be appreciated that the functional configuration shown in FIG. 2 is not intended to be limiting of server 200, and that server 200 may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 2, the memory 202, which is a kind of computer storage medium, may include therein an operating system and a remote control program of the home appliance.
In the server 200 shown in fig. 2, the processor 201 may be configured to call the remote control program stored in the memory 202 and perform the following operations:
acquiring voice information;
performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result;
generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information;
and when the reserved time is up, controlling the household appliance to operate according to the generated reserved instruction of the household appliance.
In the embodiment of the invention, the operating system of the server extracts the preset characteristic information from the acquired voice information, generates the reservation instruction and the reservation time of the household appliance according to the extracted characteristic information, and controls the household appliance to operate according to the generated reservation instruction of the household appliance when the reservation time is up. Therefore, the user only needs to send out voice information, and the advanced operation control of the household appliance can be realized through the server, so that the user operation is very convenient. In addition, the server determines the appointment time of the household appliance according to the feature information extracted from the voice information, and the user does not control the appointment time manually, so that the effective control of the advanced operation of the household appliance is realized.
In some embodiments, the server 200 further includes, for example, a voice server, and the processor 201 in the voice server calls the voice recognition program stored in the memory 202, and when acquiring the voice information, performs the following operations:
and carrying out voice recognition on the voice information, and extracting preset characteristic information from a voice recognition result. The preset characteristic information comprises departure time and a traffic mode. In addition, the preset characteristic information may also include other information, such as the time of the transaction that the user needs to process in the middle, and the like.
In some embodiments, the processor 201 in the voice server calls the voice recognition program stored in the memory 202, and when acquiring the voice information, performs the following operations:
carrying out voice recognition on the acquired voice information to obtain character information corresponding to the voice information;
and carrying out language analysis on the character information to obtain preset characteristic information.
The processor 201 converts the voice information into corresponding characters by using a voice recognition engine. And understanding and analyzing the converted characters by using a natural language understanding engine, and extracting preset characteristic information, such as parameters of analyzing the home-returning purpose, the traffic mode, the current geographic position, the time required by midway transaction processing and the like of the user. For example, in an example, the voice message sent by the user is "drive home from work now, please turn on the air conditioner in advance", the departure time is "current time", the transportation mode is "drive", the destination of the return home is "turn on the air conditioner", and the current geographical location is the company, in the extracted feature information. In another example, the voice message sent by the user is "i get off work, now drive to XX place to build body for half an hour and go home again, please turn on the water heater and the air conditioner in advance", the extracted departure time is "current time", the transportation mode is "drive", the purpose of going home is "turn on the air conditioner and the water heater", the current geographical location is company, and the time for processing the transaction halfway is "30 minutes". The above is merely an example, and the preset feature information may further include other information.
In some embodiments, the processor 201 may be configured to call a remote control program stored in the memory 202, and further perform the following operations:
estimating the distance time required to be spent between the departure place and the destination according to the transportation mode;
and determining the reservation time of the household appliance according to the starting time and the estimated journey time, and generating a reservation instruction of the household appliance.
Further, when the preset feature information further includes a time of a transaction in the middle of processing, the time of the transaction in the middle of processing is added when estimating the journey time required to be taken between the departure place and the destination.
In some embodiments, the processor 201 may be configured to call a remote control program stored in the memory 202, and further perform the following operations:
acquiring target operation parameters of the household appliance;
estimating the running time of the household appliance reaching the target running parameter according to the target running parameter of the household appliance;
and acquiring the appointment time of the household appliance according to the starting time, the estimated distance time and the running time of the household appliance, and generating an appointment instruction of the household appliance.
In some embodiments, the processor 201 may be configured to call the remote control program stored in the memory 202, and further perform the following operations before the reserved time is reached:
acquiring the movement condition of terminal control equipment in real time;
and re-determining the reserved time of the household appliance according to the movement condition of the terminal control equipment.
In some embodiments, the movement of the terminal control device comprises the geographic location and/or the movement speed of the terminal control device; the processor 201 may be configured to call the remote control program stored in the memory 202, and further perform the following operations:
according to the acquired geographic position and/or movement speed of the terminal control equipment, the journey time required to reach the destination is estimated again;
and re-determining the reservation time of the household appliance according to the re-estimated distance time.
In some embodiments, the processor 201 may be configured to call the remote control program stored in the memory 202, and further perform the following operations before the reserved time is reached:
further, when the server 200 acquires the voice information transmitted by the plurality of terminal control devices, voice recognition, extraction of preset feature information, and generation of corresponding reservation instructions and reservation times are performed, respectively. And when the first arrived reservation time arrives, controlling the household appliance to operate according to the reservation instruction corresponding to the reservation time. For example, the user a and the user B send out voice information for controlling the operation of the air conditioner in advance, and it is estimated that the user a gets home before the user B according to the voice information of the user a and the user B, so that the reservation instruction and the reservation time corresponding to the user a are earlier, and when the reservation time arrives, the household appliance is controlled to operate according to the reservation instruction corresponding to the user a.
As shown in fig. 3, fig. 3 is a schematic diagram of hardware modules of a terminal control device according to an embodiment of the present invention.
The terminal control device 100 includes a processor 101, a memory 102, a second communication module 103 for information interaction with the server 200, and an input/output module 104. The terminal control device 100 communicates with the first communication module 201 of the server 200 through the second communication module 103, so as to realize mutual communication between the terminal control device 100 and the server 200. Specifically, in an interaction example, the terminal control device 100 collects voice information, sends the collected voice information to the server 200, and the server 200 generates a corresponding reservation instruction and a reservation time according to the voice information, and controls the operation of the home appliance according to the reservation instruction when the reservation time arrives. Of course, in another interaction example, the terminal control device 100 collects voice information, performs voice recognition on the collected voice information, extracts preset feature information, generates a corresponding reservation instruction and reservation time according to the extracted feature information, and controls the operation of the home appliance device 300 according to the reservation instruction when the reservation time arrives.
In one example, the input/output module 104 includes, for example, a display screen, buttons, or a combination thereof. Of course, the input/output module 104 may also include a voice capture module to capture voice information uttered by the user.
Those skilled in the art will appreciate that the configuration shown in fig. 3 does not constitute a limitation of the terminal control device, and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.
As shown in fig. 3, the memory 102, which is a kind of computer storage medium, may include therein an operating system and a remote control program of the home appliance.
In the terminal control device shown in fig. 3, the processor 101 may call a control program stored in the memory 102, and perform the following operations:
acquiring voice information;
performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result;
generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information;
and when the reserved time is up, controlling the household appliance to operate according to the generated reserved instruction of the household appliance.
In the embodiment of the invention, the operating system of the terminal control equipment extracts the preset characteristic information from the acquired voice information, generates the reservation instruction and the reservation time of the household appliance according to the extracted characteristic information, and controls the household appliance to operate according to the generated reservation instruction of the household appliance when the reservation time is up. Therefore, the user only needs to send out voice information, the advanced operation of the household appliance can be realized, and the user operation is very convenient. In addition, the reservation time of the household appliance is determined according to the feature information extracted from the voice information, and the user does not control the reservation time manually, so that the effective control of the advanced operation of the household appliance is realized.
In some embodiments, the processor 101 may call a remote control program stored in the memory 102, and when acquiring the voice information, further perform the following operations:
and carrying out voice recognition on the voice information, and extracting preset characteristic information from a voice recognition result. The preset characteristic information comprises departure time and a traffic mode. In addition, the preset characteristic information may also include other information, such as the time of the transaction that the user needs to process in the middle, and the like.
In some embodiments, the processor 101 calls a control program stored in the memory 102, and further performs the following operations:
carrying out voice recognition on the acquired voice information to obtain character information corresponding to the voice information;
and carrying out language analysis on the character information to obtain preset characteristic information.
In some embodiments, the processor 101 may call a remote control program stored in the memory 102, and further perform the following operations:
estimating the distance time required to be spent between the departure place and the destination according to the transportation mode;
and acquiring the appointment time of the household appliance according to the starting time and the estimated distance time, and generating an appointment instruction of the household appliance.
When the preset feature information further includes a time of a transaction in the middle of processing, the time of the transaction in the middle of processing is added when estimating the journey time required to be taken between the departure place and the destination.
In some embodiments, the processor 101 may call a remote control program stored in the memory 102, and further perform the following operations:
acquiring target operation parameters of the household appliance;
estimating the running time of the household appliance reaching the target running parameter according to the target running parameter of the household appliance;
and acquiring the appointment time of the household appliance according to the starting time, the estimated distance time and the running time of the household appliance, and generating an appointment instruction of the household appliance.
In some embodiments, the processor 101 may call a remote control program stored in the memory 102, and further perform the following operations before the reserved time is reached:
acquiring the moving condition of the mobile equipment in real time;
and correcting the reserved time according to the movement condition of the mobile equipment, and re-determining the reserved time of the household appliance equipment.
According to the acquired geographic position and/or movement speed of the terminal control equipment, the journey time required to reach the destination is estimated again;
and re-determining the reservation time of the household appliance according to the re-estimated distance time.
In some embodiments, the processor 201 may be configured to call the remote control program stored in the memory 202, and further perform the following operations before the reserved time is reached:
if new voice information is acquired, performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; and re-determining the reserved time of the household appliance according to the extracted characteristic information.
In the above embodiment, the terminal control device performs voice recognition on the collected voice information by collecting the voice information sent by the user, extracts the preset feature information from the collected voice information, generates a corresponding reservation instruction and reservation time according to the preset feature information, and sends the reservation instruction to the household appliance through the server when the reservation time is reached, so as to control the operation of the household appliance. In the embodiment of the invention, the terminal control equipment is used for identifying the voice information and generating and issuing the reservation instruction, thereby lightening the burden of the server, avoiding network blockage caused by too much information and being beneficial to information communication.
In another embodiment, the terminal control device collects voice information sent by a user, performs voice recognition on the collected voice information, extracts preset feature information from the voice information, sends the preset feature information to the server, generates a corresponding reservation instruction and reservation time according to the feature information by the server, and sends the reservation instruction to the household appliance through the server when the reservation time is reached so as to control the household appliance to operate in advance. In the embodiment of the invention, the terminal control equipment is used for identifying the voice information, thereby reducing the burden of the server, avoiding network blockage caused by too much information and being beneficial to information communication.
In another embodiment, after the terminal control device collects voice information sent by a user, the voice information is sent to the voice recognition server, after the voice recognition server performs voice recognition on the voice information and extracts preset feature information, the terminal control device receives the feature information returned by the voice recognition server, generates a corresponding reservation instruction and reservation time according to the feature information, and sends the reservation instruction to the household appliance through the server when the reservation time is up to control the household appliance to operate. In the embodiment of the invention, the burden of the server is lightened, and the terminal control equipment utilizes the powerful voice recognition server to perform voice recognition, so that the accuracy of the voice recognition is improved, and the effective control of the household appliance is further improved.
The reservation command may be transmitted to the home appliance to control the operation of the home appliance when the reservation time is reached. Of course, the reservation instruction can also be sent to the household appliance together with the reservation time, and the household appliance controls the household appliance to operate according to the reservation instruction when the reservation time is up.
Further, the home appliance includes, for example, a memory, a processor, and a computer program stored in the memory and executable on the processor, where the computer program, when executed by the processor, receives a reservation instruction and a reservation time of the home appliance, and controls the home appliance to operate according to the reservation instruction of the home appliance when the reservation time arrives.
And the reservation instruction and the reservation time are sent to the household appliance by the terminal control equipment through the server. Specifically, the home appliance receives the reservation instruction and the reservation time sent by the terminal control device, and stores the reservation instruction and the reservation time. And when the reserved time is up, controlling the household appliance to operate according to the reserved instruction corresponding to the reserved time.
As shown in fig. 4, fig. 4 is a flowchart illustrating a first embodiment of a remote control method for a home appliance according to the present invention.
In this embodiment, the remote control method for a home appliance includes the steps of:
s1, acquiring voice information;
s2, performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result;
s3, generating a reservation instruction and a reservation time of the household appliance according to the extracted characteristic information;
and S4, controlling the household appliance to operate according to the generated reservation command of the household appliance when the reservation time is up.
In the embodiment of the invention, the preset characteristic information of the acquired voice information is extracted, the reservation instruction and the reservation time of the household appliance are generated according to the extracted characteristic information, and when the reservation time is up, the household appliance is controlled to operate according to the generated reservation instruction of the household appliance. Therefore, the user only needs to send out voice information to control the advanced operation of the household appliance, and the user operation is very convenient. In addition, the control time of the household appliance is determined according to the feature information extracted from the voice information and is not manually controlled by a user, so that the effective control of the advanced operation of the household appliance is realized.
In some embodiments, the voice information is collected by a terminal control device, and the terminal control device performs voice recognition on the collected voice information and extracts preset feature information from a voice recognition result. In the embodiment of the invention, the terminal control equipment is used for identifying the voice information, thereby reducing the burden of the server, avoiding network blockage caused by too much information and being beneficial to information communication.
In some embodiments, the voice information is collected by the terminal control device and sent to the voice server; and the voice information is subjected to voice recognition by the voice server, and preset characteristic information is extracted from a voice recognition result. In the embodiment of the invention, the burden of the server is lightened, and the terminal control equipment utilizes the powerful voice recognition server to perform voice recognition, so that the accuracy of the voice recognition is improved, and the effective control of the household appliance is further improved.
As shown in fig. 5, fig. 5 is a detailed flowchart of an embodiment of step S2 in fig. 4. In certain embodiments, the step S2 includes:
s21, carrying out voice recognition on the obtained voice information to obtain character information corresponding to the voice information;
and S22, carrying out language analysis on the character information to obtain preset characteristic information.
In the voice recognition process, the voice information is converted into corresponding characters by using the voice recognition engine. And understanding and analyzing the converted characters by using a natural language understanding engine, and extracting preset characteristic information, such as parameters of analyzing the home-returning purpose, the traffic mode, the current geographic position, the time required by midway transaction processing and the like of the user. For example, in an example, the voice message sent by the user is "drive home from work now, please turn on the air conditioner in advance", the departure time is "current time", the transportation mode is "drive", the destination of the return home is "turn on the air conditioner", and the current geographical location is the company, in the extracted feature information. In another example, the voice message sent by the user is "i get off work, now drive to XX place to build body for half an hour and go home again, please turn on the water heater and the air conditioner in advance", the extracted departure time is "current time", the transportation mode is "drive", the purpose of going home is "turn on the air conditioner and the water heater", the current geographical location is company, and the time for processing the transaction halfway is "30 minutes". The above is merely an example, and the preset feature information may further include other information.
In the above embodiment, through voice recognition and language parsing, extraction of feature information of complex voice information is facilitated, so that accuracy of voice recognition is improved.
As shown in fig. 6, fig. 6 is a detailed flowchart of an embodiment of step S3 in fig. 4. In certain embodiments, the step S3 includes:
s31, estimating the distance time between the departure place and the destination according to the transportation mode;
and S32, determining the reservation time of the household appliance according to the starting time and the estimated journey time, and generating a reservation instruction of the household appliance.
Specifically, a route between the departure point and the destination is determined based on the transportation mode, and the travel time between the departure point and the destination is estimated in combination with the transportation mode. The transportation mode is one or the combination of more than two of driving, taxi taking, public transportation and subway taking. Alternatively, if the traffic mode is default, for example, the voice message does not include the information of the traffic mode, or the voice message fails to extract the information of the traffic mode, the default is the previous traffic mode and the default is the estimated travel time before. For example, if the user a frequently gets home by subway or drives home after work, the route and the traffic mode of the user a are fixed, and if the voice message sent by the user a is "i want to get home after work, please turn on the air conditioner in advance", the default traffic mode of the user a getting home is "get subway" or "drive".
After the distance time is determined, the time for controlling the household appliance in advance can be determined according to the starting time, and a corresponding reservation instruction of the household appliance is generated. It should be noted that the reservation command may only include a power-on operation command, and the home appliance is automatically operated after being powered on. In addition, the reservation instruction may further include target parameters for the operation of the home appliance, such as a set temperature TS of the air conditioner, and the like.
In the embodiment of the invention, the distance time required to be spent between the departure place and the destination is estimated according to the transportation mode, so that the starting time of the household appliance can be more accurately determined.
Further, in some embodiments, the preset feature information further includes a transaction processing time in the middle. Therefore, in the estimation of the journey time, the transaction time in the middle of the journey time is added to the estimated journey time. Therefore, the control of the advanced operation of the household appliance can adapt to more situations occurring in the process of returning home of the user.
Further, as shown in fig. 7, fig. 7 is a detailed flowchart of an embodiment of step S32 in fig. 6. In certain embodiments, the step S32 specifically includes:
s321, acquiring target operation parameters of the household appliance;
s322, estimating the running time of the household appliance reaching the target running parameter according to the target running parameter of the household appliance;
and S323, determining the reservation time of the household appliance according to the starting time, the estimated distance time and the running time of the household appliance, and generating a reservation instruction of the household appliance.
The household appliance is controlled to operate in advance, and a certain time is required for the household appliance to reach the target operation parameter during operation, so that the operation time of the household appliance reaching the target operation parameter can be estimated according to the target operation parameter of the household appliance when a reservation instruction of the household appliance is generated, the estimated operation time of the household appliance is also considered when the starting time of the household appliance is determined, the household appliance is controlled to operate in advance more accurately, and a comfortable environment is provided for a user when the user returns to home.
Of course, the running time of the home device may not be estimated, but may be replaced by a preset running time. The preset run time is, for example, 8 minutes, 15 minutes, etc.
As shown in fig. 8, fig. 8 is a flowchart illustrating a second embodiment of a method for remotely controlling a home appliance according to an embodiment of the present invention. In the embodiment of the present invention, before step S4 of the above embodiment, the method further includes:
and S5, acquiring the movement condition of the terminal control equipment, and re-determining the appointment time of the household appliance according to the movement condition of the terminal control equipment.
In order to accurately determine the reserved time of the household appliance, for example, the user has a temporary situation in the process of returning home and the time of arriving home changes, in this case, the movement situation of the terminal control device is monitored, and the reserved time of the household appliance is determined again according to the motion monitoring situation of the terminal control device.
In one embodiment, the step S6 includes: according to the acquired geographic position and/or movement speed of the terminal control equipment, the journey time required to reach the destination is estimated again; and re-determining the starting time of the household appliance according to the re-estimated distance time. According to the geographic position and/or the movement speed, the fact that the user can not arrive at home on time due to traffic jam or other reasons during the process of returning home can be judged, so that the time of the user arriving at the destination is re-estimated according to the current geographic position and/or the movement speed, and the reservation time of the household appliance is re-determined.
Further, in some embodiments, before the step S4, the method further includes:
if new voice information is acquired, performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; and re-determining the reserved time of the household appliance according to the extracted characteristic information.
In this embodiment, if the user goes home and the time to go home changes, the voice message can be re-emitted. And after receiving the voice information, performing voice recognition on the voice information again, extracting preset characteristic information from the voice information, and re-determining the starting time of the household appliance according to the extracted characteristic information. Therefore, the control of the advanced operation of the household appliance is more flexible and effective.
Further, in some embodiments, when the voice information sent by the plurality of terminal control devices is acquired, voice recognition, extraction of preset feature information and generation of corresponding reservation instructions and reservation time are respectively performed; and when the first arrived reservation time arrives, controlling the household appliance to operate according to the reservation instruction corresponding to the reservation time.
When the server 200 acquires the voice information sent by the plurality of terminal control devices, voice recognition, extraction of preset feature information, and generation of corresponding reservation instructions and reservation time are performed respectively. And when the first arrived reservation time arrives, controlling the household appliance to operate according to the reservation instruction corresponding to the reservation time. For example, the user a and the user B send out voice information for controlling the operation of the air conditioner in advance, and it is estimated that the user a gets home before the user B according to the voice information of the user a and the user B, so that the reservation instruction and the reservation time corresponding to the user a are earlier, and when the reservation time arrives, the household appliance is controlled to operate according to the reservation instruction corresponding to the user a.
As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, system, or computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be noted that in the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in a claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the unit claims enumerating several means, several of these means may be embodied by one and the same item of hardware. The usage of the words first, second and third, etcetera do not indicate any ordering. These words may be interpreted as names.
While preferred embodiments of the present invention have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. Therefore, it is intended that the appended claims be interpreted as including preferred embodiments and all such alterations and modifications as fall within the scope of the invention.
It will be apparent to those skilled in the art that various changes and modifications may be made in the present invention without departing from the spirit and scope of the invention. Thus, if such modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include such modifications and variations.

Claims (18)

1. A remote control method of household appliances is characterized by comprising the following steps:
acquiring voice information;
performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; the preset characteristic information comprises departure time and a traffic mode;
generating a reservation instruction and reservation time of the household appliance according to the extracted characteristic information;
and when the reserved time is up, controlling the household appliance to operate according to the generated reserved instruction of the household appliance.
2. The remote control method of a home appliance according to claim 1, wherein the voice information is collected by a terminal control device, and the terminal control device performs voice recognition on the collected voice information and extracts preset feature information from a voice recognition result.
3. The remote control method of home appliances according to claim 1, wherein the voice information is collected by the terminal control device and transmitted to the voice server; and the voice information is subjected to voice recognition by the voice server, and preset characteristic information is extracted from a voice recognition result.
4. The remote control method of a home appliance according to claim 1, wherein the step of performing voice recognition on the acquired voice information and extracting preset feature information from a voice recognition result comprises:
carrying out voice recognition on the acquired voice information to obtain character information corresponding to the voice information;
and carrying out language analysis on the character information to obtain preset characteristic information.
5. The method of claim 1, wherein the step of generating the reservation command and the reservation time of the home appliance according to the extracted feature information comprises:
estimating the distance time required to be spent between the departure place and the destination according to the transportation mode;
and acquiring the appointment time of the household appliance according to the starting time and the estimated distance time, and generating an appointment instruction of the household appliance.
6. The method as claimed in claim 5, wherein the step of obtaining a reservation time of the home appliance and generating a reservation command of the home appliance according to the departure time and the estimated distance time comprises:
acquiring target operation parameters of the household appliance;
estimating the running time of the household appliance reaching the target running parameter according to the target running parameter of the household appliance;
and acquiring the appointment time of the household appliance according to the starting time, the estimated distance time and the running time of the household appliance, and generating an appointment instruction of the household appliance.
7. The remote control method of a home appliance device according to claim 5 or 6, wherein the preset feature information further includes a transaction time in the middle; the estimated journey time is added with the transaction processing time in the middle.
8. The method as claimed in claim 5 or 6, wherein the step of controlling the operation of the home appliance according to the generated reservation command of the home appliance when the reservation time arrives further comprises:
acquiring the movement condition of terminal control equipment in real time;
and re-determining the reserved time of the household appliance according to the movement condition of the terminal control equipment.
9. The remote control method for household electrical appliances according to claim 8, wherein the movement condition of the terminal control device comprises the geographical position and/or the movement speed of the terminal control device; the step of re-determining the reserved time of the household appliance according to the movement condition of the terminal control equipment comprises the following steps:
according to the acquired geographic position and/or movement speed of the terminal control equipment, the journey time required to reach the destination is estimated again;
and re-determining the reservation time of the household appliance according to the re-estimated distance time.
10. The method for remotely controlling a home appliance according to claim 8, wherein the step of controlling the operation of the home appliance according to the generated reservation command of the home appliance when the reservation time arrives further comprises:
if new voice information is acquired, performing voice recognition on the acquired voice information, and extracting preset characteristic information from a voice recognition result; and re-determining the reserved time of the household appliance according to the extracted characteristic information.
11. The remote control method of home appliance devices according to claim 2, wherein when acquiring the voice information transmitted from the plurality of terminal control devices, performing voice recognition, extracting preset feature information, and generating corresponding reservation instructions and reservation times, respectively; and when the first arrived reservation time arrives, controlling the household appliance to operate according to the reservation instruction corresponding to the reservation time.
12. The remote control method of an electric home appliance according to claim 1, wherein the electric home appliance comprises an air conditioning device.
13. A server, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method for remote control of an electric home appliance according to any one of claims 1 to 12, and controls the operation of the electric home appliance.
14. A terminal control device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program when executed by the processor enabling collection of voice information.
15. The terminal control device according to claim 14, wherein the computer program, when executed by the processor, further implements the steps of the method for remote control of an electric home device according to any one of claims 1 to 12.
16. The household appliance is characterized by comprising a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the computer program is used for receiving a reservation instruction and a reservation time of the household appliance when being executed by the processor, and controlling the household appliance to run according to the reservation instruction of the household appliance when the reservation time is up.
17. The home device of claim 16, wherein the reservation instruction and the reservation time of the home device are transmitted from the terminal control device to the home device through the server.
18. A computer-readable storage medium, wherein the readable storage medium has a remote control application program stored thereon, and the remote control application program is executed by a processor to implement the method for remotely controlling an electric home appliance according to any one of claims 1 to 12, and controls the electric home appliance to operate.
CN201810618496.7A 2018-06-14 2018-06-14 Household appliance and remote control method thereof, server and terminal control equipment Active CN108880957B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810618496.7A CN108880957B (en) 2018-06-14 2018-06-14 Household appliance and remote control method thereof, server and terminal control equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810618496.7A CN108880957B (en) 2018-06-14 2018-06-14 Household appliance and remote control method thereof, server and terminal control equipment

Publications (2)

Publication Number Publication Date
CN108880957A CN108880957A (en) 2018-11-23
CN108880957B true CN108880957B (en) 2021-01-08

Family

ID=64338926

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810618496.7A Active CN108880957B (en) 2018-06-14 2018-06-14 Household appliance and remote control method thereof, server and terminal control equipment

Country Status (1)

Country Link
CN (1) CN108880957B (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109634550A (en) * 2018-11-20 2019-04-16 维沃移动通信有限公司 A kind of voice operating control method and terminal device
CN110149256A (en) * 2019-07-03 2019-08-20 珠海格力电器股份有限公司 Household appliance and its control method, device
CN112575520A (en) * 2019-09-29 2021-03-30 青岛海尔洗衣机有限公司 Reservation method and device of washing equipment and storage medium
CN111338224A (en) * 2020-03-09 2020-06-26 珠海格力电器股份有限公司 Intelligent reservation control method and device based on WeChat applet and intelligent terminal
CN111571590B (en) * 2020-05-19 2023-01-06 深圳市爱康生物科技股份有限公司 Appointment control method of full-automatic sample refrigeration handover processing equipment
CN111766791A (en) * 2020-06-28 2020-10-13 青岛海尔空调器有限总公司 Method, device and equipment for controlling household equipment
CN112984732B (en) * 2021-02-20 2023-03-21 青岛海尔空调器有限总公司 Control method and device for air conditioner and air conditioner
CN112944620A (en) * 2021-02-22 2021-06-11 珠海格力电器股份有限公司 Air conditioner control method and device, storage medium and air conditioner
CN113251609B (en) * 2021-04-30 2023-03-24 青岛海尔空调器有限总公司 Control method and device for household appliance and household appliance
CN115695065A (en) * 2021-07-30 2023-02-03 青岛海尔科技有限公司 Scene creating method and device, storage medium and electronic equipment
CN116774597A (en) * 2023-05-05 2023-09-19 杭州手趣科技有限公司 Intelligent household equipment control method and system based on wearable equipment

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102316162A (en) * 2011-09-01 2012-01-11 深圳市子栋科技有限公司 Vehicle remote control method based on voice command, apparatus and system thereof
EP2843657A2 (en) * 2013-09-03 2015-03-04 Panasonic Intellectual Property Corporation of America Equipment control method and speech-based equipment control system
CN105935243A (en) * 2015-12-05 2016-09-14 肖梁坚 Automatic cooking and stewing device
CN106814716A (en) * 2017-01-03 2017-06-09 芜湖美智空调设备有限公司 The reserving method and system linked between a kind of household electrical appliances

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7602283B2 (en) * 2005-02-01 2009-10-13 American Business Solutions, Llc Active monitoring system for use with a garage door opener

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102316162A (en) * 2011-09-01 2012-01-11 深圳市子栋科技有限公司 Vehicle remote control method based on voice command, apparatus and system thereof
EP2843657A2 (en) * 2013-09-03 2015-03-04 Panasonic Intellectual Property Corporation of America Equipment control method and speech-based equipment control system
CN105935243A (en) * 2015-12-05 2016-09-14 肖梁坚 Automatic cooking and stewing device
CN106814716A (en) * 2017-01-03 2017-06-09 芜湖美智空调设备有限公司 The reserving method and system linked between a kind of household electrical appliances

Also Published As

Publication number Publication date
CN108880957A (en) 2018-11-23

Similar Documents

Publication Publication Date Title
CN108880957B (en) Household appliance and remote control method thereof, server and terminal control equipment
CN104571036B (en) Opening method and device of smart home
CN105243441B (en) Processing method and device, control method and device and electronic equipment
CN108388142A (en) Methods, devices and systems for controlling home equipment
JP5208637B2 (en) Information processing apparatus, method, and program
CN108665897B (en) Electric appliance control method and device, storage medium and server
CN108986821B (en) Method and equipment for setting relation between room and equipment
CN107703872A (en) Terminal control method, device and the terminal of home appliance
CN106707770B (en) Intelligent electrical appliance control and intelligent Home Appliance Controller
CN104896868A (en) Method for identifying and managing refrigerator food list according to remote images
CN108768806A (en) Appliances equipment control method, device, remote controler, terminal, server and medium
CN104615107B (en) The method and intelligence control system of smart home device control are carried out using mass transit card
CN111197841A (en) Control method, control device, remote control terminal, air conditioner, server and storage medium
CN113251609B (en) Control method and device for household appliance and household appliance
CN102419900A (en) Remote control realizing method and device of electrical equipment
WO2013037265A1 (en) Realization method and device for learnable type remote control
CN108833222A (en) Appliances equipment control method, device, remote controler, terminal, server and medium
CN108833223A (en) Appliances equipment control method, device, remote controler, terminal, server and medium
KR102439337B1 (en) Multilateral participation remote collaboration system based on Augmented reality sharing and method thereof
KR20170089146A (en) Method for building a driving scenarios of internet of things equipment
CN112984732B (en) Control method and device for air conditioner and air conditioner
CN104122999A (en) Intelligent device interaction method and system
CN108965081A (en) The method and device of voice control device
CN106292356B (en) A kind of robot control method and electronic equipment
CN113395193B (en) Equipment control method and device, computer equipment and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant