CN117253479A - Voice control method and system applied to wax-melting aromatherapy lamp - Google Patents
Voice control method and system applied to wax-melting aromatherapy lamp Download PDFInfo
- Publication number
- CN117253479A CN117253479A CN202311173541.XA CN202311173541A CN117253479A CN 117253479 A CN117253479 A CN 117253479A CN 202311173541 A CN202311173541 A CN 202311173541A CN 117253479 A CN117253479 A CN 117253479A
- Authority
- CN
- China
- Prior art keywords
- voice
- wax
- instruction
- wake
- aromatherapy
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000222 aromatherapy Methods 0.000 title claims abstract description 96
- 238000002844 melting Methods 0.000 title claims abstract description 79
- 238000000034 method Methods 0.000 title claims abstract description 56
- 230000008018 melting Effects 0.000 claims abstract description 34
- 229910052736 halogen Inorganic materials 0.000 claims abstract description 26
- 150000002367 halogens Chemical class 0.000 claims abstract description 26
- 230000001105 regulatory effect Effects 0.000 claims abstract description 19
- 239000002344 surface layer Substances 0.000 claims abstract description 18
- 230000005236 sound signal Effects 0.000 claims abstract description 15
- 230000001276 controlling effect Effects 0.000 claims abstract description 8
- 230000004044 response Effects 0.000 claims description 28
- 230000008451 emotion Effects 0.000 claims description 23
- 238000013519 translation Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 12
- 239000000341 volatile oil Substances 0.000 claims description 12
- 238000012545 processing Methods 0.000 claims description 9
- 230000006698 induction Effects 0.000 claims description 7
- 230000007613 environmental effect Effects 0.000 claims description 4
- 238000009432 framing Methods 0.000 claims description 4
- 239000003921 oil Substances 0.000 claims description 4
- 230000000007 visual effect Effects 0.000 claims description 4
- 238000012549 training Methods 0.000 claims description 3
- 230000009466 transformation Effects 0.000 claims description 3
- 239000003205 fragrance Substances 0.000 description 20
- 230000000694 effects Effects 0.000 description 17
- 230000009286 beneficial effect Effects 0.000 description 9
- 238000005286 illumination Methods 0.000 description 6
- 239000000796 flavoring agent Substances 0.000 description 3
- 235000019634 flavors Nutrition 0.000 description 3
- 230000036651 mood Effects 0.000 description 3
- 230000003595 spectral effect Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 238000010438 heat treatment Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 125000003118 aryl group Chemical group 0.000 description 1
- 238000004140 cleaning Methods 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000001544 dysphoric effect Effects 0.000 description 1
- 230000008909 emotion recognition Effects 0.000 description 1
- 238000004134 energy conservation Methods 0.000 description 1
- 230000005281 excited state Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 230000007246 mechanism Effects 0.000 description 1
- 210000005036 nerve Anatomy 0.000 description 1
- 238000010606 normalization Methods 0.000 description 1
- 238000001228 spectrum Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61L—METHODS OR APPARATUS FOR STERILISING MATERIALS OR OBJECTS IN GENERAL; DISINFECTION, STERILISATION OR DEODORISATION OF AIR; CHEMICAL ASPECTS OF BANDAGES, DRESSINGS, ABSORBENT PADS OR SURGICAL ARTICLES; MATERIALS FOR BANDAGES, DRESSINGS, ABSORBENT PADS OR SURGICAL ARTICLES
- A61L9/00—Disinfection, sterilisation or deodorisation of air
- A61L9/015—Disinfection, sterilisation or deodorisation of air using gaseous or vaporous substances, e.g. ozone
- A61L9/02—Disinfection, sterilisation or deodorisation of air using gaseous or vaporous substances, e.g. ozone using substances evaporated in the air by heating or combustion
- A61L9/03—Apparatus therefor
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/08—Speech classification or search
-
- H—ELECTRICITY
- H05—ELECTRIC TECHNIQUES NOT OTHERWISE PROVIDED FOR
- H05B—ELECTRIC HEATING; ELECTRIC LIGHT SOURCES NOT OTHERWISE PROVIDED FOR; CIRCUIT ARRANGEMENTS FOR ELECTRIC LIGHT SOURCES, IN GENERAL
- H05B47/00—Circuit arrangements for operating light sources in general, i.e. where the type of light source is not relevant
- H05B47/10—Controlling the light source
- H05B47/105—Controlling the light source in response to determined parameters
- H05B47/115—Controlling the light source in response to determined parameters by determining the presence or movement of objects or living beings
- H05B47/12—Controlling the light source in response to determined parameters by determining the presence or movement of objects or living beings by detecting audible sound
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61L—METHODS OR APPARATUS FOR STERILISING MATERIALS OR OBJECTS IN GENERAL; DISINFECTION, STERILISATION OR DEODORISATION OF AIR; CHEMICAL ASPECTS OF BANDAGES, DRESSINGS, ABSORBENT PADS OR SURGICAL ARTICLES; MATERIALS FOR BANDAGES, DRESSINGS, ABSORBENT PADS OR SURGICAL ARTICLES
- A61L2209/00—Aspects relating to disinfection, sterilisation or deodorisation of air
- A61L2209/10—Apparatus features
- A61L2209/11—Apparatus for controlling air treatment
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L15/00—Speech recognition
- G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
- G10L2015/223—Execution procedure of a spoken command
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Multimedia (AREA)
- Epidemiology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Toys (AREA)
Abstract
The application relates to the technical field of aromatherapy lamps, and provides a voice control method applied to a wax-melting aromatherapy lamp, which comprises the following steps: acquiring voice to be recognized in real time, wherein the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by voice input equipment; determining whether the voice to be recognized contains effective voice information by using an offline voice model; when the voice to be recognized contains an effective voice instruction, the brightness of the halogen lamp is regulated and controlled according to a control instruction corresponding to the effective voice instruction, so that the melting speed of the aromatherapy wax surface layer is changed. The invention is used for improving the convenience of controlling the wax-melting aromatherapy lamp.
Description
Technical Field
The invention relates to the technical field of intelligent home, in particular to a voice control method and a voice control system for a wax-melting aromatherapy lamp.
Background
The wax-melting aromatherapy lamp is powered by alternating voltage, and the low-melting-point aromatherapy wax is irradiated in a short distance through a high-power halogen lamp, so that the aromatherapy wax is promoted to melt and emit fragrance. Because the wax-melting aromatherapy lamp is powered by alternating current (110V-240V), the current is controlled by a knob shifting mode generally so as to control the brightness of the halogen lamp, thereby changing the irradiation light intensity of the halogen lamp, further changing the surface temperature of the aromatherapy wax and achieving the effect of adjusting the aromatic flavor or the softness of the aromatherapy wax.
However, the traditional knob shifting mode is inconvenient and flexible in use. For example, the patent application number CN201711407420.1 is a multifunctional aromatherapy machine and an assembling method thereof, the aromatherapy machine comprises a control mechanism, which comprises a first circuit board for controlling the operation of the aromatherapy machine, a first control switch for controlling the baking lamp and a second control switch for controlling the heating plate, the first control switch and the second control switch are installed outside the main casing through clamping grooves, the baking lamp and the heating plate are respectively electrically connected with the first circuit board, the above control modes are all realized by the manual control switch of a user, the control modes of the aromatherapy lamp are too complicated, the indoor fragrance effect cannot be adjusted, and the proper fragrance effect cannot be selected according to the preference of the user.
Disclosure of Invention
The invention provides a voice control method and a voice control system applied to a wax-melting aromatherapy lamp, which are used for solving the problem that the wax-melting aromatherapy lamp is not convenient enough to control.
The application provides a voice control method applied to a wax-melting aromatherapy lamp, which comprises the following steps:
acquiring voice to be recognized in real time; wherein,
the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by the voice input equipment;
determining whether the voice to be recognized contains a valid voice instruction by using an offline voice model;
when the voice to be recognized contains an effective voice instruction, the brightness of the halogen lamp is regulated and controlled according to a control instruction corresponding to the effective voice instruction, so that the melting speed of the aromatherapy wax surface layer is changed.
Preferably, before determining whether the voice to be recognized includes a valid voice instruction by using the offline voice model, the method further includes:
pre-emphasis, framing and windowing are carried out on the voice to be recognized;
extracting acoustic characteristics of the voice to be recognized after windowing; wherein,
the acoustic features include language class features and semantic features;
when the language type features are dialects, extracting tone features, pitch features and audio features to form voiceprint triples based on wavelet transformation;
determining dialect data through voiceprint triples, wherein the dialect data comprises a dialect region and a dialect voiceprint model library;
determining a first control instruction corresponding to the acoustic feature according to the dialect data;
when the language type is foreign language, extracting vocabulary characteristics, grammar characteristics and expression modes to form a translation triplet;
determining foreign language data through the translation triples;
and determining a second control instruction corresponding to the acoustic feature according to the foreign language data and based on a preset translation big model.
Preferably, the method further comprises:
presetting a multiple wake-up model, and executing a conventional wake-up mode and a scene wake-up mode; wherein,
the scene wake mode has higher priority than the conventional wake mode;
the multiple wake-up model is used for analyzing whether a conventional wake-up word and a scene wake-up word exist in the sound signal;
the conventional wake-up word is a preset wake-up word of a user;
the scene wake-up words comprise induction wake-up words under a manned scene and an unmanned scene, and the induction wake-up words comprise footstep sound and door closing sound;
identifying whether conventional wake-up words and contextual wake-up words exist in the voice to be identified by utilizing the multiple wake-up model;
if the conventional wake-up word exists, performing voice enhancement processing on the voice to be recognized after windowing processing, and separating a voice command and environmental noise;
if a scene wake-up word exists, the wax-melting aromatherapy lamp executes a default wake-up state of the corresponding scene; wherein,
the default wake-up state includes automatic wake-up in a manned scenario and default shut-down in an unmanned scenario.
Preferably, the offline speech model is a speech recognition model based on a dynamic time warping algorithm, which is specifically used for:
acquiring a sound signal in an offline state, and constructing a text time axis and word sequence control points based on the time axis;
determining instruction keywords according to the text time axis;
determining control keywords in the upper word order and the lower word order according to the instruction keywords;
constructing a word response feedback process based on the control keywords;
according to the word response feedback process, converting feedback information into starting word sequence control points of the wax-melting aromatherapy lamp;
and judging whether the wax-melting aromatherapy lamp accords with the instruction corresponding to the control keyword according to the starting word sequence control point, and performing dynamic time-ordered voice recognition.
Preferably, the method further comprises:
positioning the position prescribed by the user according to the effective voice instruction;
searching response information corresponding to the effective voice instruction from a preset response library; wherein,
the response information is the voice information fed back after receiving the effective voice instruction of the user;
playing the response information to the azimuth of the user;
when no corresponding response information exists in the instruction control library, sending out reminding information; the reminding information is information for reminding a user of replacing the control instruction.
Preferably, the offline speech model is further provided with an update procedure, and the procedure includes:
recording the effective voice instruction and configuring a corresponding offline instruction word list;
taking the real-time offline voice model as an initial offline voice model, training the offline voice model according to an offline instruction word list, and generating an updated target offline voice model;
and taking the target offline voice model as an initial offline voice model, and continuously traversing the effective voice instruction to continuously and iteratively update the offline voice model.
Preferably, the method further comprises:
recording the sending times of different effective control instructions of a user; wherein,
the effective control instruction corresponds to a scene control mode;
after a startup instruction of a user is received, when the instruction is not received within a preset time period, the most frequently used effective control instruction is used as a common control instruction, and the brightness of the halogen lamp is regulated and controlled according to the common control instruction and the corresponding scene control mode.
Preferably, the method further comprises:
determining emotion characteristics of a user according to the voice signals, and judging the current emotion state of the user; wherein,
the emotion states are divided into a low state, a medium state and a high state;
recommending at least one contextual model for the user according to the current emotion state of the user, and pushing the contextual model to the user in a voice reply mode;
and receiving a voice selection instruction of a user, and regulating and controlling the brightness of the halogen lamp according to the voice selection instruction.
Preferably, the method further comprises:
a sensing device is arranged on the wax melting aromatherapy lamp; wherein,
the sensing device is any one of a visual sensor, a weight sensor and an infrared ranging sensor;
according to the sensing device, the melting speed and the using times of the aromatherapy wax on the wax-melting aromatherapy lamp are obtained, and the loss of the essential oil in the surface layer of each aromatherapy wax is calculated;
judging whether the loss amount of the current essential oil exceeds a preset threshold value or not; wherein,
if the loss rate of the current essential oil exceeds a preset threshold, sending out alarm information to remind a user to clean wax oil on the surface layer of the aromatherapy wax.
A voice control system for a wax-melting aromatherapy lamp, comprising:
and a data acquisition module: the method is used for acquiring the voice to be recognized in real time; wherein,
the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by the voice input equipment;
and an identification module: the method comprises the steps of determining whether a voice to be recognized contains a valid voice instruction or not by using an offline voice model;
and the control module is used for: when the effective voice command is contained in the voice to be recognized, the brightness of the halogen lamp is regulated and controlled according to the control command corresponding to the effective voice command, so that the melting speed of the surface layer of the aromatherapy wax is changed.
The invention has the beneficial effects that:
whether the voice to be recognized contains an effective voice command or not is judged through the off-line voice model, and the brightness of the halogen lamp is regulated and controlled according to the control command corresponding to the effective voice command, so that the melting speed of the surface layer of the aromatherapy wax is changed, the effect of regulating the aroma emitted by the aromatherapy wax to be strong or soft is achieved, the intelligent control of the aromatherapy lamp is realized, and the use of a user is more convenient and flexible.
Additional features and advantages of the invention will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
The accompanying drawings are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate the invention and together with the embodiments of the invention, serve to explain the invention. In the drawings:
FIG. 1 is a flowchart of a voice control method applied to a wax-melting aromatherapy lamp according to an embodiment of the invention;
FIG. 2 is a flowchart of updating an offline speech module according to an embodiment of the present invention;
fig. 3 is a block diagram of a voice control system applied to a wax-melting aromatherapy lamp according to an embodiment of the invention.
Detailed Description
The preferred embodiments of the present invention will be described below with reference to the accompanying drawings, it being understood that the preferred embodiments described herein are for illustration and explanation of the present invention only, and are not intended to limit the present invention.
As shown in fig. 1, the embodiment of the invention provides a voice control method applied to a wax-melting aromatherapy lamp, which comprises the following steps:
s1: acquiring voice to be recognized in real time, wherein the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by voice input equipment;
the wax-melting aromatherapy lamp comprises a halogen lamp and aromatherapy wax. The voice input device may be a planar microphone array, where the planar microphone array can pick up a sound signal within a certain range around the wax-melting aromatherapy lamp, and convert the sound signal into an analog electric signal, and when acquiring the sound, the analog signal needs to be periodically sampled to obtain the voice to be recognized, that is, a digital voice signal. The sound source can be localized by a planar microphone array and background noise, interference, reverberation, and echo sound are suppressed. The number of the microphones can be set according to the needs, and the more the number of the microphones is, the better the voice enhancement and noise reduction effects are.
S2: determining whether the voice to be recognized contains a valid voice instruction or not by using an offline voice model;
the voice to be recognized is collected by the microphone array, the indoor environment is basically in a quiet state, and no noise is caused during voice control. When the room is noisy, for example, when a plurality of people chat or the outside sound such as a television and a sound box exists, the voice recognition is interfered, and the control of the wax-melting fragrance lamp by utilizing the voice is difficult. Therefore, it is necessary to identify the voice to be identified by using the offline voice model, and determine whether the voice to be identified contains a valid voice command, for example, the voice signal contains "on, off, etc. The offline speech model can identify whether the speech to be identified contains keywords which need to be turned on/off or turned on/off at regular time and select a contextual model.
S3: when the voice to be recognized contains an effective voice instruction, the brightness of the halogen lamp is regulated and controlled according to a control instruction corresponding to the effective voice instruction, so that the melting speed of the aromatherapy wax surface layer is changed.
The effective voice command comprises a startup and shutdown mode, a timing mode and a plurality of scene modes, each mode corresponds to different flavor effects, the illumination intensity of the halogen lamp is timely changed according to a set program by controlling the brightness of the halogen lamp and the running time of the brightness, and therefore the surface temperature of the aromatherapy wax is further changed, and the effect of adjusting the intensity or softness of the fragrance emitted by the aromatherapy wax is achieved.
The beneficial effects of the technical scheme are as follows:
whether the voice to be recognized contains an effective voice command or not is judged through the off-line voice model, and the brightness of the halogen lamp is regulated and controlled according to the control command corresponding to the effective voice command, so that the melting speed of the surface layer of the aromatherapy wax is changed, the effect of regulating the aroma emitted by the aromatherapy wax to be strong or soft is achieved, the intelligent control of the aromatherapy lamp is realized, and the use of a user is more convenient and flexible.
In one embodiment, before step S1, the method further includes:
pre-emphasis, framing and windowing are carried out on the voice to be recognized;
the pre-emphasis, framing and windowing treatment of the voice to be recognized is specifically as follows: firstly, cutting off silence at the tail end of voice receiving, and reducing interference of silence to subsequent processing steps; the voice to be recognized is emphasized through a first-order FIR high-pass digital filter, the high-frequency part in the voice to be recognized is emphasized, and the high-frequency resolution of the voice is increased; finally, the sound is framed so that there is partial overlap between successive frames.
Extracting acoustic characteristics of the voice to be recognized after windowing; wherein,
the acoustic features include language class features and semantic features; the language type features are different types of dialects or foreign languages, but the semantic features include the mood, intonation, such as prosodic features, tonal features, spectral features based on the frequency spectrum of the voice of the user.
When the language type features are dialects, extracting tone features, pitch features and audio features to form voiceprint triples based on wavelet transformation, wherein the dialects in the application are dialects based on regional features, and the expression form of the dialects is that of the language.
In practical implementation, the dialect has regional characteristics, so that language classification is performed on three characteristics of tone, pitch and audio, and dialect type positioning of voice is performed in a three-in-one mode, so that data of the dialect can be determined. Data based on regional characteristics and data characteristics based on temperature rise.
The dialect data is determined by voiceprint triples, wherein,
the dialect data comprises a dialect region and a dialect voiceprint model library;
determining a first control instruction corresponding to the acoustic feature according to the dialect data;
when the language type is foreign language, extracting vocabulary characteristics, grammar characteristics and expression modes to form a translation triplet; the foreign language in the present application is a language other than open chinese, including other national languages such as english and languages of minority nations, for example, tibetan.
When the control voice is a foreign language, because the foreign language is different from the regional characteristics, but the language architecture mode is different, when analysis is performed, the mode of forming the triples is based on the vocabulary characteristics, the grammar characteristics and the expression mode, so that specific foreign language data of the whole foreign language is formed. The foreign language data realizes the recognition of acoustic features through a translation large model, and the translation large model is a language large model which is trained in advance to realize language translation and has the functions of translation and the analysis of voice and the normalization of pronunciation modes.
Determining foreign language data through the translation triples;
and determining a second control instruction corresponding to the acoustic feature according to the foreign language data and based on a preset translation big model.
The beneficial effects of the technical scheme are as follows:
by extracting the acoustic characteristics of the voice to be recognized, the language type of the user is recognized, and the user can control the wax-melting aromatherapy lamp by using the conventional language or dialect. The invention can identify various language types, realizes automatic identification of the language types, provides comfortable and safe atmosphere for users, and better serves the users.
In one embodiment, a multiple wake-up model is preset, and a conventional wake-up mode and a scene wake-up mode are executed; wherein,
the scene wake mode has higher priority than the conventional wake mode;
the multiple wake-up model is used for analyzing whether a conventional wake-up word and a scene wake-up word exist in the sound signal;
the conventional wake-up word is a preset wake-up word of a user;
the scene wake-up words comprise induction wake-up words under a manned scene and an unmanned scene, and the induction wake-up words comprise footstep sound and door closing sound;
identifying whether conventional wake-up words and contextual wake-up words exist in the voice to be identified by utilizing the multiple wake-up model;
if the conventional wake-up word exists, performing voice enhancement processing on the voice to be recognized after windowing processing, and separating a voice command and environmental noise;
if a scene wake-up word exists, the wax-melting aromatherapy lamp executes a default wake-up state of the corresponding scene; wherein,
the default wake-up state includes automatic wake-up in a manned scenario and default shut-down in an unmanned scenario.
The principle of the technical scheme is as follows:
in this application, the effect of multiple model of awakening is in order to realize melting wax fragrance lamp's energy-conservation and saving wax, and conventional voice control equipment is because just can consume the electric energy, so awaken up through conventional mode and just can reach control effect, but to the product of this application can have wax still to have the lamp, if melt wax and illumination for a long time under unmanned state, very consume electric energy and wax.
The present application is when performing the normal wake-up mode:
the wake-up words can be identified through the multiple wake-up model, and different illumination wax melting modes can be identified, under the condition that the conventional wake-up words are keywords mainly based on effective voice instructions, such as turning off a lamp after one hour, turning off a lamp after six hours, a fragrance mode, a leisure mode, a night lamp mode and the like, the illuminance and the time of each mode control are different, such as the leisure mode, and the effect of firstly enriching and then slowing down is achieved through the control of the halogen lamp.
The multiple wake-up model can monitor the voice of the user in real time, and when a starting instruction of the user is received, the voice control of the wax-melting aromatherapy lamp is started, and a subsequent voice instruction is received. When the voice wake-up word is set, whether noise exists in the current room or not can be determined through different voice wake-up words, for example, the wake-up word can be a small A when the voice wake-up word is quiet, the wake-up word is a small B when the voice wake-up word is noisy, when a user receives the wake-up word when the voice wake-up word is noisy, voice enhancement is performed on voice to be recognized after windowing processing, and a voice command and environmental noise are separated. The voice enhancement can remove the voice signal of the background noise, the voice enhancement is specifically: noise estimation is performed first by spectral subtraction, and then filtering is performed.
When the contextual wake mode is executed, the method comprises the following steps:
the real-time azimuth of the user is judged by sensing the footstep sound and the door closing sound, so that whether the user is still indoors or not is judged, if the user is indoors, illumination and wax melting are continuously carried out, indoor illumination and air freshness are ensured, and the door is automatically closed when no person is involved; and automatically turned on in the presence of a person to receive a conventional wake-up word.
The beneficial effects of the technical scheme are as follows:
the household appliances which are interacted manually are many, so that false starting is prevented; and confirm whether the speech to be recognized needs denoising processing at present through different wake-up words, can also open and close the wax-melting aromatherapy lamp according to the state of someone and nobody to reduce the electric power loss and the loss of wax.
In one embodiment, the off-line speech model is a speech recognition model based on a dynamic time warping algorithm, which is specifically used for:
acquiring a sound signal in an offline state, and constructing a text time axis and word sequence control points based on the time axis;
determining instruction keywords according to the text time axis;
determining control keywords in the upper word order and the lower word order according to the instruction keywords;
constructing a word response feedback process based on the control keywords;
according to the word response feedback process, converting feedback information into starting word sequence control points of the wax-melting aromatherapy lamp;
and judging whether the wax-melting aromatherapy lamp accords with the instruction corresponding to the control keyword according to the starting word sequence control point, and performing dynamic time-ordered voice recognition.
The principle of the technical scheme is as follows:
the text time axis corresponds to the text and the time point.
The instruction keywords comprise keywords representing control attributes such as aromatherapy, wax melting, time and the like, and a response process is constructed for each instruction keyword based on the constructed text time axis and word sequence control points corresponding to each control keyword, so that whether the wax melting aromatherapy lamp executes instructions of a user is judged.
Specifically, when the instruction keywords are included in the user's sentences, for example, a fragrance lamp is turned on or a wax melting lamp is turned off, and when fragrance/wax melting is detected, control keywords in the upper word sequence and the lower word sequence are determined according to the instruction keywords, wherein the control keywords comprise starting, closing, awakening words, mode selection and the like.
The word response feedback process realizes direct reaction by corresponding each textified control word to the specific execution information of the wax-melting aromatherapy lamp, and the dynamic time warping is based on the dynamic time warping of each control word to determine each word of the voice recognition.
The beneficial effects of the technical scheme are as follows:
the method and the device determine the instruction keywords through the textualization time axis, determine the control keywords of the upper word order and the lower word order according to the instruction keywords after the instruction keywords are detected, timely make feedback, finally convert feedback information into the lighted word order control points, and perform dynamic time warping according to whether the word order control points are lighted or not. The offline voice model can detect and analyze the instruction of the user in real time, and when detecting that the voice signal contains the control keywords, the offline voice model can immediately feed back the control keywords, and has high response speed and efficiency.
In one embodiment, the method further comprises the steps of:
positioning the position prescribed by the user according to the effective voice instruction;
searching response information corresponding to the effective voice instruction from a preset response library; wherein,
the response information is the voice information fed back after receiving the effective voice instruction of the user;
playing the response information to the azimuth of the user;
when no corresponding response information exists in the instruction control library, sending out reminding information; the reminding information is information for reminding a user of replacing the control instruction.
The principle of the technical scheme is as follows:
according to the method and the device, the user positioning based on the microphone array can be performed according to the effective voice command, the position of the user is determined, the response information is played to the position, the authenticity of man-machine conversation is improved, and the emotion requirement of the user is cared for.
The response information includes: the received and opened fragrance mode, good, owner, etc. can select the response words with different emotion degrees according to the user's liking.
When no corresponding response information exists in the instruction control library, the reminding information is sent out, wherein the reminding information is information for reminding a user to replace the control instruction, for example, please speak again.
Specifically, the offline speech model is further provided with an update process, and the process includes:
recording the effective voice instruction and configuring a corresponding offline instruction word list;
taking the real-time offline voice model as an initial offline voice model, training the offline voice model according to an offline instruction word list, and generating an updated target offline voice model;
and taking the target offline voice model as an initial offline voice model, and continuously traversing the effective voice instruction to continuously and iteratively update the offline voice model.
The principle of the technical scheme is as follows:
as shown in fig. 2, when the offline speech model of the present application is in a network non-offline state, an effective speech command is recorded at any time and automatically converted into an offline command vocabulary, and continuous updating iteration is performed on the offline speech model through the offline command vocabulary.
The beneficial effects of the technical scheme are as follows:
the off-line voice model is updated once at intervals, the functions of the wax-melting aromatherapy lamp are increased, and interaction is better performed.
Specifically, the method further comprises the following steps:
recording the sending times of different effective control instructions of a user; wherein,
the effective control instruction corresponds to a scene control mode;
after a startup instruction of a user is received, when the instruction is not received within a preset time period, the most frequently used effective control instruction is used as a common control instruction, and the brightness of the halogen lamp is regulated and controlled according to the common control instruction and the corresponding scene control mode.
The principle of the technical scheme is as follows:
and determining a scene model according to the preference degree of the user for each mode, which is equivalent to personalized recommendation, and when the user cannot make a decision, selecting a common mode for the user to better serve the user.
Specifically, the method further comprises the following steps:
determining emotion characteristics of a user according to the voice signals, and judging the current emotion state of the user; wherein,
the emotion states are divided into a low state, a medium state and a high state;
recommending at least one contextual model for the user according to the current emotion state of the user, and pushing the contextual model to the user in a voice reply mode;
and receiving a voice selection instruction of a user, and regulating and controlling the brightness of the halogen lamp according to the voice selection instruction.
The principle of the technical scheme is as follows:
because the freshness of air and the light of warm color system have the effect of leveling the mood of a user, the emotion state of the user can be judged, the contextual model is adjusted according to the emotion state, and the brightness, the melting speed of the fragrance wax and the expression mode of the self voice of the wax-melting fragrance lamp are improved, so that the mood of the user is leveled.
In this process, the emotion acoustic features with high relevance are extracted to help determine the emotion state of a speaker, and the acoustic features are usually extracted in units of frames, but the features are generally used as inputs of a model in a global statistical mode to participate in emotion recognition. The acoustic features of the voice to be recognized comprise prosodic features, spectral features, tone quality features, human ear auditory characteristics and the like, the current emotion state of the user is determined through the features, and the low-correspondence user is in sad, angry and dysphoric states; indicating that the user is in a calm state; a high indicates that the user is in a happy, excited state.
Recommending at least one contextual model for the user according to the emotion state of the user, and pushing the contextual model to the user in a voice reply mode;
the nerve soothing effect required by the user in different emotion states is different, a plurality of current required contextual models are recommended to the user according to the states of the user, and the experience of the user is improved.
The beneficial effects of the technical scheme are as follows:
and confirming the emotion state of the user according to the current acoustic characteristics of the user, recommending at least one mode matched with the current emotion state for the user, and selecting the mode for the user, so that the comfort of the user can be improved, and the mind and body of the user can be relieved.
In one embodiment, the method further comprises:
a sensing device is arranged on the wax melting aromatherapy lamp; wherein,
the sensing device is any one of a visual sensor, a weight sensor and an infrared ranging sensor;
in the application, the visual sensor can judge the melting speed and the using times of the aromatherapy wax in an image recognition mode, so that the loss is calculated; the weight sensor judges the melting speed and the using times of the aromatherapy wax through weight induction of the aromatherapy wax, so that the loss is calculated; the infrared ranging sensor measures the concentration of the essential oil and the height of the aromatherapy wax through ranging, so that the melting speed and the using times of the aromatherapy wax are judged, and the loss is calculated.
According to the sensing device, the melting speed and the using times of the aromatherapy wax on the wax-melting aromatherapy lamp are obtained, and the loss of the essential oil in the surface layer of each aromatherapy wax is calculated;
judging whether the loss amount of the current essential oil exceeds a preset threshold value or not; wherein,
if the loss rate of the current essential oil exceeds a preset threshold, sending out alarm information to remind a user to clean wax oil on the surface layer of the aromatherapy wax.
The beneficial effects of the technical scheme are as follows:
judging whether the essential oil in the aromatherapy wax surface layer is about to be invalid or not according to the loss rate of the current essential oil and a preset threshold value, and sending out alarm information when the loss rate exceeds the preset threshold value to remind a user of cleaning wax oil on the aromatherapy wax surface layer.
A voice control system for a wax-melting aromatherapy lamp, comprising:
and a data acquisition module: the method is used for acquiring the voice to be recognized in real time; wherein,
the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by the voice input equipment;
and an identification module: the method comprises the steps of determining whether a voice to be recognized contains a valid voice instruction or not by using an offline voice model;
and the control module is used for: when the effective voice command is contained in the voice to be recognized, the brightness of the halogen lamp is regulated and controlled according to the control command corresponding to the effective voice command, so that the melting speed of the surface layer of the aromatherapy wax is changed.
The principle of the technical scheme is as follows:
as shown in fig. 3, the data acquisition module of the present application may acquire a voice signal, where the voice signal is acquired based on a microphone array in a wax-melting fragrance lamp, and the wax-melting fragrance lamp of the present application includes a halogen lamp and fragrance wax. The voice input device of the wax-melting aromatherapy lamp can be a planar microphone array, the planar microphone array can pick up voice signals in a certain range around the wax-melting aromatherapy lamp and convert the voice signals into analog electric signals, and when the voice is acquired, the analog signals are required to be periodically sampled to obtain voice to be recognized, namely digital voice signals. The sound source can be localized by a planar microphone array and background noise, interference, reverberation, and echo sound are suppressed. The number of the microphones can be set according to the needs, and the more the number of the microphones is, the better the voice enhancement and noise reduction effects are.
The recognition module is used for voice recognition, namely instruction recognition of a user, in the process, voice to be recognized is collected by the microphone array, the indoor environment is basically in a quiet state, and noise is not influenced during voice control. When the room is noisy, for example, when a plurality of people chat or the outside sound such as a television and a sound box exists, the voice recognition is interfered, and the control of the wax-melting fragrance lamp by utilizing the voice is difficult. Therefore, it is necessary to identify the voice to be identified by using the offline voice model, and determine whether the voice to be identified contains a valid voice command, for example, the voice signal contains "on, off, etc. The offline speech model can identify whether the speech to be identified contains keywords which need to be turned on/off or turned on/off at regular time and select a contextual model.
The control module receives the effective voice command, the effective voice command comprises a startup and shutdown mode, a timing mode and a plurality of scene modes, each mode corresponds to different brightness (can also comprise the flavor of the fragrance lamp) effects, the illumination light intensity of the halogen lamp is timely changed according to a set program through the control of the brightness of the halogen lamp and the running time of the brightness, and therefore the surface temperature of the fragrance wax is further changed, and the effect of adjusting the fragrance emitted by the fragrance wax to be strong or soft is achieved.
The beneficial effects of the technical scheme are as follows:
whether the voice to be recognized contains an effective voice instruction or not is determined, and the brightness of the halogen lamp is regulated and controlled by utilizing a control instruction corresponding to the effective voice instruction, so that the melting speed of the surface layer of the aromatherapy wax is changed, the effect of regulating the aroma emitted by the aromatherapy wax to be strong or soft is achieved, the intelligent control of the aromatherapy lamp is realized, and the use of a user is more convenient and flexible.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (10)
1. A voice control method applied to a wax-melting aromatherapy lamp, which is characterized by comprising the following steps:
acquiring voice to be recognized in real time; wherein,
the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by the voice input equipment;
determining whether the voice to be recognized contains a valid voice instruction by utilizing a voice model; wherein,
the voice model comprises an offline voice model and an online voice model;
when the voice to be recognized contains an effective voice instruction, the brightness of the halogen lamp is regulated and controlled according to a control instruction corresponding to the effective voice instruction, so that the melting speed of the aromatherapy wax surface layer is changed.
2. The method of claim 1, wherein before determining whether the voice to be recognized includes a valid voice command by using the offline voice model, further comprises:
pre-emphasis, framing and windowing are carried out on the voice to be recognized;
extracting acoustic characteristics of the voice to be recognized after windowing; wherein,
the acoustic features include language class features and semantic features;
when the language type features are dialects, extracting tone features, pitch features and audio features to form voiceprint triples based on wavelet transformation;
determining dialect data through voiceprint triples, wherein the dialect data comprises a dialect region and a dialect voiceprint model library;
determining a first control instruction corresponding to the acoustic feature according to the dialect data;
when the language type is foreign language, extracting vocabulary characteristics, grammar characteristics and expression modes to form a translation triplet;
determining foreign language data through the translation triples;
and determining a second control instruction corresponding to the acoustic feature according to the foreign language data and based on a preset translation big model.
3. The method of claim 2, wherein the method further comprises:
presetting a multiple wake-up model, and executing a conventional wake-up mode and a scene wake-up mode; wherein,
the scene wake mode has higher priority than the conventional wake mode;
the multiple wake-up model is used for analyzing whether a conventional wake-up word and a scene wake-up word exist in the sound signal;
the conventional wake-up word is a preset wake-up word of a user;
the scene wake-up words comprise induction wake-up words under a manned scene and an unmanned scene, and the induction wake-up words comprise footstep sound and door closing sound;
identifying whether conventional wake-up words and contextual wake-up words exist in the voice to be identified by utilizing the multiple wake-up model;
if the conventional wake-up word exists, performing voice enhancement processing on the voice to be recognized after windowing processing, and separating a voice command and environmental noise;
if a scene wake-up word exists, the wax-melting aromatherapy lamp executes a default wake-up state of the corresponding scene; wherein,
the default wake-up state includes automatic wake-up in a manned scenario and default shut-down in an unmanned scenario.
4. The method of claim 1, wherein the offline speech model is a speech recognition model based on a dynamic time warping algorithm, and is specifically used for:
acquiring a sound signal in an offline state, and constructing a text time axis and word sequence control points based on the time axis;
determining instruction keywords according to the text time axis;
determining control keywords in the upper word order and the lower word order according to the instruction keywords;
constructing a word response feedback process based on the control keywords;
according to the word response feedback process, converting feedback information into starting word sequence control points of the wax-melting aromatherapy lamp;
and judging whether the wax-melting aromatherapy lamp accords with the instruction corresponding to the control keyword according to the starting word sequence control point, and performing dynamic time-ordered voice recognition.
5. The method of claim 1, further comprising:
positioning the position prescribed by the user according to the effective voice instruction;
searching response information corresponding to the effective voice instruction from a preset response library; the response information is voice information fed back after receiving an effective voice instruction of the user;
playing the response information to the azimuth of the user;
when no corresponding response information exists in the instruction control library, sending out reminding information; the reminding information is information for reminding a user of replacing the control instruction.
6. The voice control method for the wax-melting aromatherapy lamp according to claim 1, wherein the offline voice model is further provided with an update procedure, the procedure comprises:
recording the effective voice instruction and configuring a corresponding offline instruction word list;
taking the real-time offline voice model as an initial offline voice model, training the offline voice model according to an offline instruction word list, and generating an updated target offline voice model;
and taking the target offline voice model as an initial offline voice model, and continuously traversing the effective voice instruction to continuously and iteratively update the offline voice model.
7. A method of voice control for a wax-melting aromatherapy lamp as in claim 3, further comprising:
recording the sending times of different effective control instructions of a user; wherein,
the effective control instruction corresponds to a scene control mode;
after a startup instruction of a user is received, when the instruction is not received within a preset time period, the most frequently used effective control instruction is used as a common control instruction, and the brightness of the halogen lamp is regulated and controlled according to the common control instruction and the corresponding scene control mode.
8. The method of claim 2, wherein the method further comprises:
determining emotion characteristics of a user according to the voice signals, and judging the current emotion state of the user; wherein,
the emotion states are divided into a low state, a medium state and a high state;
recommending at least one contextual model for the user according to the current emotion state of the user, and pushing the contextual model to the user in a voice reply mode;
and receiving a voice selection instruction of a user, and regulating and controlling the brightness of the halogen lamp according to the voice selection instruction.
9. The method of claim 1, further comprising:
a sensing device is arranged on the wax melting aromatherapy lamp; wherein,
the sensing device is any one of a visual sensor, a weight sensor and an infrared ranging sensor;
according to the sensing device, the melting speed and the using times of the aromatherapy wax on the wax-melting aromatherapy lamp are obtained, and the loss of the essential oil in the surface layer of each aromatherapy wax is calculated;
judging whether the loss amount of the current essential oil exceeds a preset threshold value or not; wherein,
if the loss rate of the current essential oil exceeds a preset threshold, sending out alarm information to remind a user to clean wax oil on the surface layer of the aromatherapy wax.
10. A voice control system applied to a wax-melting aromatherapy lamp is characterized by comprising a data acquisition module, an identification module and a control module, wherein:
and a data acquisition module: the method is used for acquiring the voice to be recognized in real time; wherein,
the voice to be recognized is a sound signal around the wax melting aromatherapy lamp picked up by the voice input equipment;
and an identification module: the method comprises the steps of determining whether a voice to be recognized contains a valid voice instruction or not by using an offline voice model;
and the control module is used for: when the effective voice command is contained in the voice to be recognized, the brightness of the halogen lamp is regulated and controlled according to the control command corresponding to the effective voice command, so that the melting speed of the surface layer of the aromatherapy wax is changed.
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311173541.XA CN117253479A (en) | 2023-09-12 | 2023-09-12 | Voice control method and system applied to wax-melting aromatherapy lamp |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202311173541.XA CN117253479A (en) | 2023-09-12 | 2023-09-12 | Voice control method and system applied to wax-melting aromatherapy lamp |
Publications (1)
Publication Number | Publication Date |
---|---|
CN117253479A true CN117253479A (en) | 2023-12-19 |
Family
ID=89136122
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202311173541.XA Pending CN117253479A (en) | 2023-09-12 | 2023-09-12 | Voice control method and system applied to wax-melting aromatherapy lamp |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN117253479A (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118098237A (en) * | 2024-04-24 | 2024-05-28 | 武汉攀升鼎承科技有限公司 | Control method of intelligent voice mouse and intelligent voice mouse |
Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN105126146A (en) * | 2015-07-24 | 2015-12-09 | 江西智能无限物联科技有限公司 | Intelligent aromatherapy device and intelligent control system |
CN108882454A (en) * | 2018-07-20 | 2018-11-23 | 佛山科学技术学院 | A kind of intelligent sound identification interaction means of illumination and system based on emotion judgment |
CN109545197A (en) * | 2019-01-02 | 2019-03-29 | 珠海格力电器股份有限公司 | Voice instruction identification method and device and intelligent terminal |
US10388277B1 (en) * | 2015-06-25 | 2019-08-20 | Amazon Technologies, Inc. | Allocation of local and remote resources for speech processing |
CN110884806A (en) * | 2019-11-29 | 2020-03-17 | 潘江 | Infrared wisdom keg of microwave sound wave intelligence response pronunciation answer light |
CN111968645A (en) * | 2020-09-09 | 2020-11-20 | 重庆电子工程职业学院 | Personalized voice control system |
CN112305927A (en) * | 2019-07-31 | 2021-02-02 | 深圳市云海物联科技有限公司 | Equipment control method and device |
CN112885346A (en) * | 2021-01-19 | 2021-06-01 | 安徽迪科数金科技有限公司 | Intelligent language identification and translation system |
CN116386623A (en) * | 2023-02-28 | 2023-07-04 | 青岛海尔科技有限公司 | Voice interaction method of intelligent equipment, storage medium and electronic device |
-
2023
- 2023-09-12 CN CN202311173541.XA patent/CN117253479A/en active Pending
Patent Citations (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10388277B1 (en) * | 2015-06-25 | 2019-08-20 | Amazon Technologies, Inc. | Allocation of local and remote resources for speech processing |
CN105126146A (en) * | 2015-07-24 | 2015-12-09 | 江西智能无限物联科技有限公司 | Intelligent aromatherapy device and intelligent control system |
CN108882454A (en) * | 2018-07-20 | 2018-11-23 | 佛山科学技术学院 | A kind of intelligent sound identification interaction means of illumination and system based on emotion judgment |
CN109545197A (en) * | 2019-01-02 | 2019-03-29 | 珠海格力电器股份有限公司 | Voice instruction identification method and device and intelligent terminal |
CN112305927A (en) * | 2019-07-31 | 2021-02-02 | 深圳市云海物联科技有限公司 | Equipment control method and device |
CN110884806A (en) * | 2019-11-29 | 2020-03-17 | 潘江 | Infrared wisdom keg of microwave sound wave intelligence response pronunciation answer light |
CN111968645A (en) * | 2020-09-09 | 2020-11-20 | 重庆电子工程职业学院 | Personalized voice control system |
CN112885346A (en) * | 2021-01-19 | 2021-06-01 | 安徽迪科数金科技有限公司 | Intelligent language identification and translation system |
CN116386623A (en) * | 2023-02-28 | 2023-07-04 | 青岛海尔科技有限公司 | Voice interaction method of intelligent equipment, storage medium and electronic device |
Cited By (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN118098237A (en) * | 2024-04-24 | 2024-05-28 | 武汉攀升鼎承科技有限公司 | Control method of intelligent voice mouse and intelligent voice mouse |
CN118098237B (en) * | 2024-04-24 | 2024-07-02 | 武汉攀升鼎承科技有限公司 | Control method of intelligent voice mouse and intelligent voice mouse |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220317641A1 (en) | Device control method, conflict processing method, corresponding apparatus and electronic device | |
CN108320733B (en) | Voice data processing method and device, storage medium and electronic equipment | |
Vacher et al. | Development of audio sensing technology for ambient assisted living: Applications and challenges | |
CN108882454B (en) | Intelligent voice recognition interactive lighting method and system based on emotion judgment | |
CN109189980A (en) | The method and electronic equipment of interactive voice are carried out with user | |
CN117253479A (en) | Voice control method and system applied to wax-melting aromatherapy lamp | |
KR20010113919A (en) | Method of interacting with a consumer electronics system | |
CN112113317B (en) | Indoor thermal environment control system and method | |
CN109429416A (en) | Illumination control method, apparatus and system for multi-user scene | |
CN112201233A (en) | Voice control method, system and device of intelligent household equipment and computer storage medium | |
CN109429415A (en) | Illumination control method, apparatus and system | |
JP2009087074A (en) | Equipment control system | |
WO2023185005A1 (en) | Working mode switching method and apparatus | |
CN109235590A (en) | Intelligent closestool control method and intelligent closestool | |
CN105700359A (en) | Method and system for controlling smart home through speech recognition | |
CN109164713A (en) | Intelligent household control method and device | |
CN109859752A (en) | Voice control method, device, storage medium and voice joint control system | |
CN114863932A (en) | Working mode setting method and device | |
CN114999472A (en) | Air conditioner control method and device and air conditioner | |
WO2019221894A1 (en) | Intelligent device user interactions | |
CN1494053A (en) | Speaking person standarding method and speech identifying apparatus using the same | |
US11133020B2 (en) | Assistive technology | |
WO2023185007A1 (en) | Sleep scene setting method and apparatus | |
CN116386623A (en) | Voice interaction method of intelligent equipment, storage medium and electronic device | |
CN115175415A (en) | Digital twinning light adjusting method, device and system |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |