CN115104548A - Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology - Google Patents
Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology Download PDFInfo
- Publication number
- CN115104548A CN115104548A CN202210809103.7A CN202210809103A CN115104548A CN 115104548 A CN115104548 A CN 115104548A CN 202210809103 A CN202210809103 A CN 202210809103A CN 115104548 A CN115104548 A CN 115104548A
- Authority
- CN
- China
- Prior art keywords
- pet
- audio
- frame
- energy
- heart rate
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A01—AGRICULTURE; FORESTRY; ANIMAL HUSBANDRY; HUNTING; TRAPPING; FISHING
- A01K—ANIMAL HUSBANDRY; CARE OF BIRDS, FISHES, INSECTS; FISHING; REARING OR BREEDING ANIMALS, NOT OTHERWISE PROVIDED FOR; NEW BREEDS OF ANIMALS
- A01K29/00—Other apparatus for animal husbandry
- A01K29/005—Monitoring or measuring activity, e.g. detecting heat or mating
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
- A61B5/0205—Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
- A61B5/02055—Simultaneously evaluating both cardiovascular condition and temperature
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B5/00—Measuring for diagnostic purposes; Identification of persons
- A61B5/74—Details of notification to user or communication with user or patient ; user input means
- A61B5/746—Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/09—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being zero crossing rates
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
- G10L25/24—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters the extracted parameters being the cepstrum
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B2503/00—Evaluating a particular growth phase or type of persons or animals
- A61B2503/40—Animals
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Life Sciences & Earth Sciences (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Multimedia (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Acoustics & Sound (AREA)
- Physiology (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Cardiology (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Surgery (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Environmental Sciences (AREA)
- Business, Economics & Management (AREA)
- General Physics & Mathematics (AREA)
- Pulmonology (AREA)
- Animal Husbandry (AREA)
- Biodiversity & Conservation Biology (AREA)
- Emergency Management (AREA)
- Measuring And Recording Apparatus For Diagnosis (AREA)
Abstract
The invention relates to the field of intelligent pet education, in particular to a pet behavior adjustment and human/pet interaction method and device based on a multimedia information technology, which comprises the following steps: receiving pet behavior monitoring instructions, starting monitoring equipment to shoot pet video images, utilizing heart rate sensor and temperature sensor to test pet heart rate and temperature value, and judging whether pet heart rate and temperature value meet health conditions, if meet and continue to execute the monitoring instructions, if not satisfy and start the sound collection unit and acquire pet audio, after pre-emphasis, framing windowing, endpoint detection processing, extracting pet audio characteristic parameters to classify into the pre-constructed filed pet sound set, according to the classification result, judging the pet demand that the pet audio reflects, based on at last the reward punishment instructions that pet video images and APP remote terminal sent accomplish pet behavior adjustment and human pet interaction. The invention can solve the problems of insufficient monitoring of pet health and lack of judgment of pet requirements of the current intelligent interaction device.
Description
Technical Field
The invention relates to the field of intelligent pet education, in particular to a pet behavior adjustment and human/pet interaction method and device based on a multimedia information technology.
Background
Along with the improvement of living standard and the increase of living pressure, more and more people begin to raise the pet, but the demand of work makes owner can't accompany in real time at the pet, has prompted out the interactive technique of various intelligent pets and device from this, contains automatic feeder, remote monitoring equipment, electronic toy. Although automatic feeder can realize that the pet feeds the function, and remote monitoring equipment can know the pet state through establishing video conversation, and electronic toy can accompany the pet and play, but above three kinds of intelligent pet interactive technology and device are difficult to let owner real-time, know pet health status and pet demand under network off-line state very much, have that health monitoring is not enough, the problem that the pet demand lacks the judgement.
Disclosure of Invention
The invention provides a pet behavior adjustment and human pet interaction method and device based on a multimedia information technology and a computer readable storage medium, and mainly aims to solve the problems that a master is insufficient in pet health monitoring and needs are lack of judgment.
In order to achieve the above object, the present invention provides a pet behavior adjustment and human pet interaction method based on multimedia information technology, comprising:
receiving a pet behavior monitoring instruction sent by an APP remote terminal, and starting a sound acquisition unit and a sensor unit in a pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
testing the heart rate of the pet by using the heart rate sensor, and acquiring a pet temperature value of the temperature sensor;
judging whether the pet heart rate and the pet temperature value meet health conditions, if so, continuing to execute the pet behavior monitoring instruction, and if not, starting the sound acquisition unit;
acquiring initial audio of the sound acquisition unit, and obtaining frame windowing audio through pre-emphasis and frame windowing processing;
carrying out end point monitoring on the frame windowing audio and extracting characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
will pet video image that supervisory equipment acquireed extremely APP remote terminal receives the reward punishment instruction that APP remote terminal sent, based on reward punishment instruction takes corresponding reward punishment means, realizes pet action adjustment and human pet interdynamic.
Optionally, said testing the pet heart rate with said heart rate sensor comprises:
starting the heart rate sensor, wherein the heart rate sensor comprises four elements of a light emitting diode, a photodiode, a photosensitive sensor and an AD converter;
the light emitting diode is used for emitting light waves to the skin of the pet, so that the light waves irradiate into the skin of the pet;
receiving the residual light waves reflected back by the skin of the pet through the photodiode, and converting the residual light waves into electric signals by using the photosensitive sensor;
and converting the electric signal into a digital signal based on the AD converter to obtain the heart rate of the pet.
Optionally, the obtaining of the initial audio of the sound collection unit and obtaining the frame-wise windowed audio through pre-emphasis and frame-wise windowing processing includes:
constructing a pre-emphasis processing function, wherein the pre-emphasis processing function is as follows:
y(n)＝x(n)-αx(n-1)
wherein y (n) represents the pet weighted audio obtained after the pre-emphasis processing at the moment n, alpha represents the pre-emphasis coefficient and takes a value of 0.98, and x (n) represents the pet initial audio obtained by the sound acquisition unit at the moment n;
processing an initial audio by utilizing the pre-emphasis processing function to obtain the emphasized audio;
and performing frame windowing on the emphasized audio to obtain frame windowed audio.
Optionally, the frame windowing processes the emphasized audio to obtain a frame windowed audio, including:
setting the frame length value of the weighted audio frame of the pet;
performing framing operation on the emphasized audio based on python to obtain pet framed audio;
constructing a windowing function to perform windowing operation on the pet framed audio to obtain the framed windowed audio, wherein the windowing function is as follows:
wherein w (B) is the windowing function, B is the window length corresponding to the frame length value of the pet weighted audio framing, the specific value is B, and n is the moment of the pet framing audio.
Optionally, the performing endpoint monitoring on the framed windowed audio and extracting the characteristic parameters of the framed windowed audio includes:
constructing a short-time energy function to calculate the short-time energy of the frame-wise windowed audio, wherein the short-time energy function is as follows:
wherein E is n Representing the short-term energy of each frame of the frame-windowed audio of the pet, K is the frame length value of the frame-weighted audio frame of the pet, x t Representing the tth frame of the pet frame windowed audio;
based on a preset short-time energy threshold value, filtering a mute part in the frame windowing audio to obtain a pet energy audio;
calculating the short-time zero crossing rate of the pet energy audio, and further filtering a mute section in the pet energy audio to obtain a pet endpoint audio;
and extracting the characteristic parameters of the pet endpoint audio by using a characteristic extraction function.
Optionally, the calculating a short-time zero crossing rate of the pet energy audio, and further filtering a silence segment in the pet energy audio to obtain a pet endpoint audio includes:
constructing the following function to calculate the short-time zero crossing rate of the pet energy audio:
wherein Z is n Is the short-time zero-crossing rate, x, of the pet energy audio t Representing the t frame of the pet frame with the window audio, K is the frame length value of the pet weighted audio frame, sgn (x) t ) Is x t A step function of (a);
setting a zero crossing rate threshold value of a mute section;
and filtering the pet energy audio with the short-time zero crossing rate out of the threshold range of the zero crossing rate of the mute section to obtain the pet endpoint audio.
Optionally, the extracting, by using a feature extraction function, feature parameters of the pet endpoint audio includes:
carrying out fast Fourier transform on the pet endpoint audio based on python to obtain a pet energy frequency spectrum;
smoothing the pet energy frequency spectrum and eliminating harmonic wave operation by utilizing M triangular filters to obtain a pet smooth energy frequency spectrum;
calculating logarithmic energy of the smooth energy spectrum of the pet output by the M triangular filters;
performing discrete cosine transform on the logarithmic energy to obtain a standard MFCC parameter;
calculating a first order difference and a second order difference of the standard MFCC parameters to obtain dynamic MFCC parameters;
and summarizing the standard MFCC parameters and the dynamic MFCC parameters to generate the N-dimensional MFCC characteristic parameters of the pet endpoint audio.
Optionally, the extracting, by using a feature extraction function, the feature parameter of the pet endpoint audio includes:
carrying out fast Fourier transform on the pet endpoint audio based on python to obtain a pet energy frequency spectrum;
smoothing the pet energy frequency spectrum and eliminating harmonic wave operation by utilizing M triangular filters to obtain a pet smooth energy frequency spectrum;
calculating logarithmic energy of the smooth energy spectrum of the pet output by the M triangular filters;
performing discrete cosine transform on the logarithmic energy to obtain a standard MFCC parameter;
calculating a first-order difference and a second-order difference of the standard MFCC parameters to obtain dynamic MFCC parameters;
and summarizing the standard MFCC parameters and the dynamic MFCC parameters to generate the N-dimensional MFCC characteristic parameters of the pet endpoint audio.
Optionally, the discrete cosine transforming the logarithmic energy to obtain a standard MFCC parameter includes:
constructing the following model to perform discrete cosine transform on the logarithmic energy:
wherein, c (B) is an MFCC parameter, s (M) is logarithmic energy of the pet smooth energy spectrum, B is a window length corresponding to a frame length value of the pet weighted audio frame, specifically takes a value of B, M is the number of triangular filters, specifically takes a value of M, and L is an MFCC parameter;
the standard MFCC parameters are calculated with the value of the above formula L set to 1.
In order to solve the above problems, the present invention further provides a pet behavior adjustment and human/pet interaction device based on multimedia information technology, the device comprising:
the monitoring instruction receiving module is used for receiving a pet behavior monitoring instruction sent by the APP remote terminal and starting a sound acquisition unit and a sensor unit in the pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
the heart rate temperature acquisition module is used for testing the heart rate of the pet by using the heart rate sensor and acquiring the pet temperature value of the temperature sensor;
the health state judging module is used for judging whether the pet heart rate and the pet temperature value meet health conditions or not, if the pet heart rate and the pet temperature value meet the health conditions, the pet behavior monitoring instruction is continuously executed, and if the pet heart rate and the pet temperature value do not meet the health conditions, the sound collecting unit is started;
the audio processing module is used for acquiring the initial audio of the sound acquisition unit and obtaining the frame windowing audio through pre-emphasis and frame windowing processing;
the pet demand judging module is used for carrying out end point monitoring on the frame windowing audio and extracting the characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
the pet interactive module is used for synchronizing pet video images acquired by the monitoring equipment to the APP remote terminal and receiving the reward punishment instruction sent by the APP remote terminal, and based on the reward punishment instruction, corresponding reward punishment means are taken to realize pet behavior adjustment and pet interaction.
In order to solve the above problem, the present invention also provides an electronic device, including:
a memory storing at least one instruction; and
and the processor executes the instructions stored in the memory to realize the pet behavior adjustment and human/pet interaction method based on the multimedia information technology.
In order to solve the above problem, the present invention further provides a computer-readable storage medium, where at least one instruction is stored in the computer-readable storage medium, and the at least one instruction is executed by a processor in an electronic device to implement the pet behavior adjustment and human pet interaction method based on multimedia information technology described above.
In order to solve the problems in the background art, after a pet behavior monitoring instruction sent by an APP remote terminal is received, a sound collecting unit and a sensor unit in a pet collar are started according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor, whether the pet heart rate tested by the heart rate sensor and the pet temperature obtained by the temperature sensor meet health conditions or not is judged in real time, if the pet heart rate and the pet temperature meet the health conditions, the pet behavior monitoring instruction is continuously executed, and the problem that the current intelligent interaction technology and device are insufficient in pet health monitoring is solved. If the pet heart rate and the pet temperature value do not satisfy the health condition, starting the sound collection unit to obtain initial pet audio, obtaining the pet endpoint audio after pre-emphasis, framing and windowing, and endpoint detection processing, classifying the pet endpoint audio into a pre-constructed file pet sound set based on characteristic parameters extracted from the pet endpoint audio, judging pet requirements reflected by the pet endpoint audio according to classification results, if the pet hospitalization requirements are reflected, sending hospitalization warning information to an APP remote terminal to remind a pet owner of carrying the pet to seek medical attention in time, if the pet accompanying requirements are reflected, receiving a human/pet interaction instruction sent by the APP remote terminal, starting a loudspeaker in a pet collar, constructing voice conversation connection of the APP remote terminal and a pet receiving terminal, and realizing pet behavior adjustment and human/pet interaction, therefore, the problem that the current intelligent interaction technology and device lack judgment on pet requirements is effectively solved.
Drawings
FIG. 1 is a schematic flow chart illustrating a pet behavior adjustment and human/pet interaction method based on multimedia information technology according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart showing a detailed implementation of one of the steps in FIG. 1;
FIG. 3 is a schematic flow chart showing another step of FIG. 1;
FIG. 4 is a functional block diagram of a pet behavior modification and human/pet interaction device based on multimedia messaging technology according to an embodiment of the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device for implementing the pet behavior adjustment and human/pet interaction method based on the multimedia information technology according to an embodiment of the present invention.
The implementation, functional features and advantages of the objects of the present invention will be further explained with reference to the accompanying drawings.
Detailed Description
It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The embodiment of the application provides a pet behavior adjustment and human/pet interaction method based on a multimedia information technology. The implementation subject of the pet behavior adjustment and human pet interaction method based on multimedia information technology includes, but is not limited to, at least one of a server, a terminal, and other electronic devices that can be configured to implement the method provided in the embodiments of the present application. In other words, the pet behavior adjustment and human pet interaction method based on the multimedia information technology can be executed by software or hardware installed in the terminal device or the server device. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.
Referring to fig. 1, a flow chart of a pet behavior adjustment and human/pet interaction method based on a multimedia information technology according to an embodiment of the present invention is shown. In this embodiment, the pet behavior adjustment and human pet interaction method based on the multimedia information technology includes:
s1, receiving a pet behavior monitoring instruction sent by the APP remote terminal, and starting a sound acquisition unit and a sensor unit in the pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor.
It can be explained that the APP remote terminal can be any terminal that can install the pet behavior adjustment and human/pet interaction device based on the multimedia information technology according to the embodiment of the present invention, such as a mobile phone of a pet owner, a desktop computer, a smart watch, and the like.
What need understand is that pet owner sends pet action monitoring instruction through APP remote terminal, mainly contains pet health status monitoring and judges according to the pet demand that the pet audio frequency goes on, wherein pet health status monitoring is based on pet rhythm of the heart, the temperature sensor pet temperature value that heart rate sensor obtained and is judged, the audio frequency judgement that the pet demand judgement that goes on according to the pet audio frequency acquireed through the sound acquisition unit.
S2, testing the heart rate of the pet by using the heart rate sensor, and acquiring the pet temperature value of the temperature sensor.
In detail, referring to fig. 2, the testing the pet heart rate by using the heart rate sensor includes:
s21, starting the heart rate sensor, wherein the heart rate sensor comprises four elements of a light emitting diode, a photodiode, a photosensitive sensor and an AD converter;
s22, emitting light waves to the pet skin by using the light emitting diode, and enabling the light waves to irradiate the pet skin;
s23, receiving the residual light waves reflected by the pet skin through the photodiode, and converting the residual light waves into electric signals by using the photosensitive sensor;
and S24, converting the electric signal into a digital signal based on the AD converter, and obtaining the heart rate of the pet.
It is clear that the light emitted by the led towards the pet's skin is green, flashing hundreds of times per second, the reason for using green light is that pet's blood is red, reflecting red light, but absorbing green light. At the moment that the heart of the pet beats, the amount of blood circulating at the neck of the pet is increased, the absorbed green light is more, the absorbed green light is less at the heart beat interval of the pet, and the heart rate of the pet is further judged by judging the absorption condition of the blood of the pet to the green light.
It should be explained that the green light is absorbed by the muscle, the bone and the blood of the pet after passing through the skin of the pet neck, and in order to ensure the accuracy of the heart rate measurement of the pet, the part of the green light absorbed by the muscle and the bone of the pet needs to be excluded. For a specific pet, the green light absorbed by the muscle and the skeleton of the pet is constant, and only the green light absorbed by the blood is changed. Therefore, after the photodiode receives the residual green light reflected by the skin of the pet, the embodiment of the invention further converts the residual green light into an electric signal by using the photosensitive sensor, and the electric signal converted by the photosensitive sensor is divided into two types, namely a direct current DC signal and an alternating current AC signal, wherein the direct current DC signal reflects the green light condition absorbed by the muscle and the skeleton of the pet, and the alternating current AC signal reflects the green light condition absorbed by the blood of the pet.
Further, the influence of the direct current DC signal is eliminated, and the alternating current AC signal is converted into a digital signal by the AD converter, namely the pet heart rate is obtained.
S3, judging whether the pet heart rate and the pet temperature value meet health conditions, if so, continuing to execute the pet behavior monitoring instruction, and if not, starting the sound collection unit.
Illustratively, the healthy heart rate value for an adult pet cat should range from one hundred twenty to one hundred eighty times per minute, and if the heart rate sensor calculates that the adult pet cat heart rate is in a healthy range, then the heart rate of the adult pet cat continues to be monitored, and if the heart rate sensor calculates that the adult pet cat heart rate is not in a healthy range, then the sound collection unit within the pet collar is activated. Similarly, the healthy temperature range of the adult pet cat should be 38 degrees to 39 degrees, if the temperature of the adult pet cat obtained by the temperature sensor is in the healthy range, the temperature of the adult pet cat is continuously monitored, and if the adult pet cat temperature calculated by the temperature sensor is not in the healthy range, the sound collection unit in the pet collar is started.
And S4, acquiring the initial audio of the sound acquisition unit, and obtaining the frame windowing audio through pre-emphasis and frame windowing processing.
In detail, referring to fig. 3, the obtaining of the initial audio of the sound collection unit, and the obtaining of the frame-wise windowed audio through pre-emphasis and frame-wise windowing includes:
s41, constructing a pre-emphasis processing function, wherein the pre-emphasis processing function is as follows:
y(n)＝x(n)-αx(n-1)
wherein y (n) represents the pet weighted audio obtained after the pre-emphasis processing at the moment n, alpha represents the pre-emphasis coefficient and takes a value of 0.98, and x (n) represents the pet initial audio obtained by the sound acquisition unit at the moment n;
s42, processing the initial audio by utilizing the pre-emphasis processing function to obtain the emphasis audio;
and S43, performing frame windowing processing on the emphasized audio to obtain frame windowed audio.
Further, the frame windowing processes the emphasized audio to obtain a frame windowed audio, including:
setting the frame length value of the weighted audio framing of the pet;
performing framing operation on the emphasized audio based on python to obtain pet framed audio;
constructing a windowing function to perform windowing operation on the pet framed audio to obtain the framed windowed audio, wherein the windowing function is as follows:
wherein w (B) is the windowing function, B is the window length corresponding to the frame length value of the pet weighted audio framing, the specific value is B, and n is the moment of the pet framing audio.
It should be noted that the frame length value is usually set to 256 or 512, and the time length of each frame of audio of the pet weighted audio is 20-30 ms, for example, if the frame length value is set to 256 and the frequency of the pet weighted audio is 8KHZ, the time length of each frame of the pet weighted audio is 256/8000-1000-32 ms. The frame length value set by the embodiment of the invention is 256, and the time length of each frame of the obtained pet frame audio is 32 ms.
It can be seen that, when the pet endpoint audio is subjected to the subsequent fast fourier transform, a certain assumed condition exists, that is, the signal of the pet endpoint audio is a periodic signal, and if the signal of the pet endpoint audio is not a periodic signal, a distorted frequency spectrum is obtained when the fast fourier transform is performed on the object endpoint audio, and a spectrum leakage effect occurs, so that the windowing operation is performed on the framed audio, so that the framed audio becomes a framed windowed audio with a periodic signal.
It should be understood that the windowing operation is to multiply the frame audio by the windowing function, where B is the window length corresponding to the frame length value, and the frame length value set in the embodiment of the present invention is 256, and the window length B takes the value of 240.
S5, carrying out end point monitoring on the frame windowing audio and extracting characteristic parameters of the frame windowing audio; and classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results.
In detail, the performing endpoint monitoring on the framed windowed audio and extracting the characteristic parameters of the framed windowed audio includes:
constructing a short-time energy function to calculate the short-time energy of the frame-wise windowed audio, wherein the short-time energy function is as follows:
wherein E is n Representing the short-term energy of each frame of the frame-windowed audio of the pet, K is the frame length value of the frame-weighted audio frame of the pet, x t Representing the tth frame of the pet frame windowed audio;
based on a preset short-time energy threshold value, filtering a mute part in the frame windowing audio to obtain a pet energy audio;
calculating the short-time zero crossing rate of the pet energy audio, and further filtering a mute section in the pet energy audio to obtain a pet endpoint audio;
and extracting the characteristic parameters of the pet endpoint audio by using a characteristic extraction function.
Wherein, the short-time zero crossing rate of the pet energy audio is calculated, and the silence section in the pet energy audio is further filtered to obtain the pet endpoint audio, and the method comprises the following steps:
constructing the following function to calculate the short-time zero crossing rate of the pet energy audio:
wherein Z is n Is the short-time zero-crossing rate, x, of the pet energy audio t Representing the frame t of the pet frame, K is the frame length value of the pet frame with the weighted audio frame, sgn (x) t ) Is x t A step function of (a);
setting a zero crossing rate threshold value of a mute section;
and filtering the pet energy audio with the short-time zero crossing rate out of the threshold range of the zero crossing rate of the mute section to obtain the pet endpoint audio.
Further, the extracting the feature parameters of the pet endpoint audio by using the feature extraction function includes:
carrying out fast Fourier transform on the pet endpoint audio based on python to obtain a pet energy frequency spectrum;
smoothing the pet energy frequency spectrum and eliminating harmonic wave operation by utilizing M triangular filters to obtain a pet smooth energy frequency spectrum;
calculating logarithmic energy of the smooth energy spectrum of the pet output by the M triangular filters;
performing discrete cosine transform on the logarithmic energy to obtain a standard MFCC parameter;
calculating a first-order difference and a second-order difference of the standard MFCC parameters to obtain dynamic MFCC parameters;
and summarizing the standard MFCC parameters and the dynamic MFCC parameters to generate the N-dimensional MFCC characteristic parameters of the pet endpoint audio.
In addition, the calculating the logarithmic energy of the smoothed energy spectrum of the pet output by the M triangular filters includes:
the following model is constructed to calculate the logarithmic energy of the smoothed energy spectrum of the pet:
wherein S (m) is the logarithmic energy of the smoothed energy spectrum of the pet, x t Representing the frame t of the pet framed windowing audio, K is the frame length value of the pet weighted audio framed, M is the number of the triangular filters, and the specific value is M, and H (M) is the frequency of the M triangular filters.
Further, the discrete cosine transforming the logarithmic energy to obtain a standard MFCC parameter includes:
constructing the following model to perform discrete cosine transform on the logarithmic energy:
wherein, c (B) is an MFCC parameter, s (M) is logarithmic energy of the pet smooth energy spectrum, B is a window length corresponding to a frame length value of the pet weighted audio frame, specifically takes a value of B, M is the number of triangular filters, specifically takes a value of M, and L is an MFCC parameter;
the standard MFCC parameters are calculated with the value of the above formula L set to 1.
It should be explained that, the end point detection is performed on the framed windowed audio, and the mute section without the pet audio in the framed windowed audio needs to be deleted. The first step is to calculate the short-time energy of the frame-wise windowed audio of the pet based on the short-time energy function, the preset short-time energy threshold value is 120-140 in the embodiment of the invention, and the short-time energy of the frame-wise windowed audio is not in the range of the preset short-time energy threshold value, which is regarded as a mute part and needs to be deleted. The second step is to calculate the short-time zero-crossing rate of the pet energy audio (the number of times that the voice signal changes from positive to negative or negative to positive in each frame of audio, namely the number of times that the voice signal passes through zero) based on a short-time zero-crossing rate function, set the zero-crossing rate threshold value of the mute section to be 5-10, filter the pet energy audio with the short-time zero-crossing rate outside the zero-crossing rate threshold value range of the mute section, and obtain the pet endpoint audio containing the non-mute pet audio.
Understandably, the voice signal changes in the time dimension, which is usually difficult to see the characteristics of the signal, and needs to be converted into energy distribution in the frequency domain, and a Fourier transform command can be constructed through python, so that the pet endpoint audio is converted into a pet energy spectrum in the frequency spectrum. In the fourier transform command, a fourier transform point number needs to be set, and the set fourier transform point number in the embodiment of the present invention is 128 corresponding to the sampling frequency of each frame of audio, that is, 128 samples are performed in each frame of the pet endpoint audio.
It should be noted that, harmonic waves with the tone being several times higher than other tones exist in the pet energy spectrum, the harmonic waves cannot accurately measure the MFCC parameters, and smoothing processing is required. Wherein the value of M is 26.
The method can be explained by that a pre-constructed file pet sound set extracts characteristic parameters of different types of audio of pets through pre-emphasis, framing and windowing and endpoint detection operations of different types of audio of pets collected in advance, and asks professional pet doctors to judge pet requirements of different types of audio responses. After the characteristic parameters of the frame windowed audio are obtained through calculation, the frame windowed audio is matched and classified into a pre-constructed archived pet sound set according to the principle that the characteristic parameters are the same and similar, and the pet requirements reflected by the frame windowed audio can be judged according to the classification result.
It is clear that need be, if the pet demand is the demand of seeking medical advice, sends warning information of seeking medical advice to APP remote terminal, reminds pet owner to carry the pet in time to seek medical advice, if the pet demand is the companion demand, receives the people's pet interactive instruction that APP remote terminal sent, opens speaker in the pet neck ring constructs the voice conversation of APP remote terminal and pet receiving terminal is connected.
S6, will the pet video image that supervisory equipment acquireed extremely APP remote terminal, receive the reward punishment instruction that APP remote terminal sent, based on reward punishment instruction takes corresponding reward punishment means, realize pet action adjustment and human pet interdynamic.
It should be noted that the monitoring device is installed in a room where a pet can reach, for example, if the pet can reach a bedroom and a living room, the monitoring device needs to be installed in both the bedroom and the living room. After the pet behavior monitoring instruction is received, the monitoring equipment starts to shoot video images of the pet, and synchronizes the video images to the APP remote terminal in real time. The pet owner passes through the video image of pet is looked over to APP remote terminal to send reward punishment instruction, exemplary, if video image is that the pet dog is tearing open the family, then pet dog owner passes through APP remote terminal sends punishment instruction, at this moment, the pet neck ring begins vibrations, corrects pet dog's mistake action of tearing open the family, if video image is that the pet dog waits for owner's home at lambkin, then pet dog owner can pass through APP remote terminal sends the reward instruction, at this moment, the dog bone snack that adorns in advance in the pet neck ring will pop out automatically to realize pet action adjustment and people's pet interdynamic.
In order to solve the problems in the background art, after a pet behavior monitoring instruction sent by an APP remote terminal is received, a sound collecting unit and a sensor unit in a pet collar are started according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor, whether the pet heart rate tested by the heart rate sensor and the pet temperature obtained by the temperature sensor meet health conditions or not is judged in real time, if the pet heart rate and the pet temperature meet the health conditions, the pet behavior monitoring instruction is continuously executed, and the problem that the current intelligent interaction technology and device are insufficient in pet health monitoring is solved. If the pet heart rate and the pet temperature value do not satisfy the health condition, starting the sound collection unit to obtain initial pet audio, obtaining the pet endpoint audio after pre-emphasis, framing and windowing, and endpoint detection processing, classifying the pet endpoint audio into a pre-constructed file pet sound set based on characteristic parameters extracted from the pet endpoint audio, judging pet requirements reflected by the pet endpoint audio according to classification results, if the pet hospitalization requirements are reflected, sending hospitalization warning information to an APP remote terminal to remind a pet owner of carrying the pet to seek medical attention in time, if the pet accompanying requirements are reflected, receiving a human/pet interaction instruction sent by the APP remote terminal, starting a loudspeaker in a pet collar, constructing voice conversation connection of the APP remote terminal and a pet receiving terminal, and realizing pet behavior adjustment and human/pet interaction, therefore, the problem that the current intelligent interaction technology and device lack judgment on pet requirements is effectively solved.
Fig. 4 is a functional block diagram of a pet behavior adjustment and human/pet interaction device based on multimedia information technology according to an embodiment of the present invention.
The pet behavior adjustment and human/pet interaction device 100 based on the multimedia information technology according to the present invention may be installed in an electronic device. According to the realized function, the pet behavior adjustment and pet interaction device 100 based on the multimedia information technology may include a monitoring instruction receiving module 101, a heart rate and temperature obtaining module 102, a health status determining module 103, an audio processing module 104, a pet requirement determining module 105, and a pet interaction module 106. The module of the present invention, which may also be referred to as a unit, refers to a series of computer program segments that can be executed by a processor of an electronic device and can perform a fixed function, and are stored in a memory of the electronic device.
The monitoring instruction receiving module 101 is used for receiving a pet behavior monitoring instruction sent by the APP remote terminal, and starting a sound collecting unit and a sensor unit in a pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
the heart rate temperature acquisition module 102 is configured to test a heart rate of a pet by using the heart rate sensor, and acquire a pet temperature value of the temperature sensor;
the health state judgment module 103 is configured to judge whether the pet heart rate and the pet temperature value meet health conditions, continue to execute the pet behavior monitoring instruction if the pet heart rate and the pet temperature value meet the health conditions, and start the sound collection unit if the pet heart rate and the pet temperature value do not meet the health conditions;
the audio processing module 104 is configured to obtain an initial audio of the sound acquisition unit, and obtain a framed windowed audio through pre-emphasis and frame windowing;
the pet demand judging module 105 is configured to perform endpoint monitoring on the framed windowed audio and extract characteristic parameters of the framed windowed audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
the human pet interaction module 106 is used for synchronizing the pet video images acquired by the monitoring equipment to the APP remote terminal and receiving the reward punishment instruction sent by the APP remote terminal, and based on the reward punishment instruction, corresponding reward punishment means are taken to realize pet behavior adjustment and human pet interaction.
In detail, the specific implementation manner of using each module in the pet behavior adjustment and human/pet interaction device 100 based on the multimedia information technology in the embodiment of the present invention is the same as that in embodiment 1, and is not repeated herein.
Fig. 5 is a schematic structural diagram of an electronic device for implementing pet behavior adjustment and human/pet interaction methods based on a multimedia information technology according to an embodiment of the present invention.
The electronic device 1 may include a processor 10, a memory 11, and a bus 12, and may further include a computer program stored in the memory 11 and executable on the processor 10, such as a pet behavior adjustment and human pet interaction method program based on multimedia information technology.
The memory 11 includes at least one type of readable storage medium, which includes flash memory, removable hard disk, multimedia card, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disk, optical disk, etc. The memory 11 may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used to store not only application software installed in the electronic device 1 and various data, such as codes of pet behavior adjustment and pet interaction method programs based on multimedia information technology, etc., but also temporarily store data that has been output or will be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device, connects various components of the electronic device by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules (e.g., pet behavior adjustment and human pet interaction method programs based on multimedia information technology, etc.) stored in the memory 11 and calling data stored in the memory 11.
The bus 12 may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus 12 may be divided into an address bus, a data bus, a control bus, etc. The bus 12 is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
Fig. 5 only shows an electronic device with components, and it will be understood by a person skilled in the art that the structure shown in fig. 5 does not constitute a limitation of the electronic device 1, and may comprise fewer or more components than shown, or a combination of certain components, or a different arrangement of components.
For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), and optionally a standard wired interface, a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The pet behavior adjustment and human pet interaction method program based on multimedia information technology stored in the memory 11 of the electronic device 1 is a combination of a plurality of instructions, and when running in the processor 10, can implement:
receiving a pet behavior monitoring instruction sent by an APP remote terminal, and starting a sound acquisition unit and a sensor unit in a pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
testing the heart rate of the pet by using the heart rate sensor, and acquiring a pet temperature value of the temperature sensor;
judging whether the pet heart rate and the pet temperature value meet health conditions, if so, continuing to execute the pet behavior monitoring instruction, and if not, starting the sound acquisition unit;
acquiring initial audio of the sound acquisition unit, and obtaining frame windowing audio through pre-emphasis and frame windowing processing;
carrying out end point monitoring on the frame windowing audio and extracting characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
will pet video image that supervisory equipment acquireed extremely APP remote terminal receives the reward punishment instruction that APP remote terminal sent, based on reward punishment instruction takes corresponding reward punishment means, realizes pet action adjustment and human pet interdynamic.
Specifically, the specific implementation method of the processor 10 for the instruction may refer to the description of the relevant steps in the embodiments corresponding to fig. 1 to fig. 5, which is not repeated herein.
Further, the integrated modules/units of the electronic device 1 may be stored in a computer-readable storage medium if they are implemented in the form of software functional units and sold or used as separate products. The computer readable storage medium may be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
The present invention also provides a computer-readable storage medium, storing a computer program which, when executed by a processor of an electronic device, may implement:
receiving a pet behavior monitoring instruction sent by an APP remote terminal, and starting a sound acquisition unit and a sensor unit in a pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
testing the heart rate of the pet by using the heart rate sensor, and acquiring a pet temperature value of the temperature sensor;
judging whether the pet heart rate and the pet temperature value meet health conditions, if so, continuing to execute the pet behavior monitoring instruction, and if not, starting the sound acquisition unit;
acquiring initial audio of the sound acquisition unit, and obtaining frame windowing audio through pre-emphasis and frame windowing processing;
carrying out end point monitoring on the frame windowing audio and extracting characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
will pet video image that supervisory equipment acquireed extremely APP remote terminal receives the reward punishment instruction that APP remote terminal sent, based on reward punishment instruction takes corresponding reward punishment means, realizes pet action adjustment and human pet interdynamic.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus, device and method can be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or means recited in the system claims may also be implemented by one unit or means in software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A pet behavior adjustment and human pet interaction method based on a multimedia information technology is characterized by comprising the following steps:
receiving a pet behavior monitoring instruction sent by an APP remote terminal, and starting monitoring equipment, a sound acquisition unit and a sensor unit in a pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
testing the heart rate of the pet by using the heart rate sensor, and acquiring a pet temperature value of the temperature sensor;
judging whether the pet heart rate and the pet temperature value meet health conditions, if so, continuing to execute the pet behavior monitoring instruction, and if not, starting the sound acquisition unit;
acquiring initial audio of the sound acquisition unit, and obtaining frame windowing audio through pre-emphasis and frame windowing processing;
carrying out end point monitoring on the frame windowing audio and extracting characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
and synchronizing the pet video images acquired by the monitoring equipment to the APP remote terminal, receiving reward and punishment instructions sent by the APP remote terminal, and realizing pet behavior adjustment and human pet interaction.
2. The method of claim 1, wherein the step of testing the heart rate of the pet by using the heart rate sensor comprises:
starting the heart rate sensor, wherein the heart rate sensor comprises four elements of a light emitting diode, a photodiode, a photosensitive sensor and an AD converter;
the light emitting diode is used for emitting light waves to the skin of the pet, so that the light waves irradiate into the skin of the pet;
receiving the residual light waves reflected back by the skin of the pet through the photodiode, and converting the residual light waves into electric signals by using the photosensitive sensor;
and converting the electric signal into a digital signal based on the AD converter to obtain the heart rate of the pet.
3. The method of claim 1, wherein the obtaining of the initial audio of the sound collection unit and the obtaining of the framed windowed audio through pre-emphasis and framed windowed processing comprises:
constructing a pre-emphasis processing function, wherein the pre-emphasis processing function is as follows:
y(n)＝x(n)-αx(n-1)
wherein y (n) represents the pet weighted audio obtained after the pre-emphasis processing at the moment n, alpha represents the pre-emphasis coefficient and takes a value of 0.98, and x (n) represents the pet initial audio obtained by the sound acquisition unit at the moment n;
processing an initial audio by utilizing the pre-emphasis processing function to obtain the emphasized audio;
and performing frame windowing on the emphasized audio to obtain frame windowed audio.
4. The method of claim 3, wherein the framing windowing the emphasized audio to obtain a framed windowed audio comprises:
setting the frame length value of the weighted audio framing of the pet;
performing framing operation on the emphasized audio based on python to obtain pet framed audio;
constructing a windowing function to perform windowing operation on the pet framed audio to obtain the framed windowed audio, wherein the windowing function is as follows:
wherein w (B) is the windowing function, B is the window length corresponding to the frame length value of the pet weighted audio framing, the specific value is B, and n is the moment of the pet framing audio.
5. The method of claim 1, wherein the monitoring the endpoints of the windowed audio frame and extracting the characteristic parameters of the windowed audio frame comprises:
constructing a short-time energy function to calculate the short-time energy of the frame-wise windowed audio, wherein the short-time energy function is as follows:
wherein E is n Representing the short-term energy of each frame of the frame-windowed audio of the pet, K is the frame length value of the frame-weighted audio frame of the pet, x t Representing the tth frame of the pet frame windowed audio;
based on a preset short-time energy threshold value, filtering a mute part in the frame windowing audio to obtain a pet energy audio;
calculating the short-time zero crossing rate of the pet energy audio, and further filtering a mute section in the pet energy audio to obtain a pet endpoint audio;
and extracting the characteristic parameters of the pet endpoint audio by using a characteristic extraction function.
6. The method of claim 5, wherein the calculating a short-term zero-crossing rate of the pet energy audio and further filtering a silence segment of the pet energy audio to obtain a pet endpoint audio comprises:
constructing the following function to calculate the short-time zero crossing rate of the pet energy audio:
wherein Z is n Is the short-time zero crossing rate, x, of the pet energy audio t Representing the frame t of the pet frame, K is the frame length value of the pet frame with the weighted audio frame, sgn (x) t ) Is x t A step function of (2);
setting a zero crossing rate threshold value of a mute section;
and filtering the pet energy audio with the short-time zero-crossing rate out of the threshold range of the zero-crossing rate of the mute section to obtain the pet endpoint audio.
7. The method of claim 5, wherein the extracting characteristic parameters of the pet endpoint audio using a characteristic extraction function comprises:
carrying out fast Fourier transform on the pet endpoint audio based on python to obtain a pet energy frequency spectrum;
smoothing the pet energy frequency spectrum and eliminating harmonic wave operation by utilizing M triangular filters to obtain a pet smooth energy frequency spectrum;
calculating logarithmic energy of the smooth energy spectrum of the pet output by the M triangular filters;
performing discrete cosine transform on the logarithmic energy to obtain a standard MFCC parameter;
calculating a first-order difference and a second-order difference of the standard MFCC parameters to obtain dynamic MFCC parameters;
and summarizing the standard MFCC parameters and the dynamic MFCC parameters to generate the N-dimensional MFCC characteristic parameters of the pet endpoint audio.
8. The method of claim 7, wherein the computing log energy of the smoothed energy spectrum of the pet output by the M triangular filters comprises:
the following model is constructed to calculate the logarithmic energy of the smoothed energy spectrum of the pet:
wherein S (m) is the logarithmic energy of the smoothed energy spectrum of the pet, x t Representing the frame t of the pet framed windowing audio, K is the frame length value of the pet weighted audio framed, M is the number of the triangular filters, and the specific value is M, and H (M) is the frequency of the M triangular filters.
9. The method of claim 7, wherein the discrete cosine transforming the logarithmic energy to obtain standard MFCC parameters comprises:
constructing the following model to perform discrete cosine transform on the logarithmic energy:
wherein, c (B) is an MFCC parameter, s (M) is logarithmic energy of the pet smooth energy spectrum, B is a window length corresponding to a frame length value of the pet weighted audio frame, specifically taking a value of B, M is the number of the triangular filters, specifically taking a value of M, and L is an MFCC parameter;
the standard MFCC parameters are calculated with the value of the above formula L set to 1.
10. A pet behavior adjustment and human pet interaction device based on multimedia information technology is characterized in that the device comprises:
the monitoring instruction receiving module is used for receiving a pet behavior monitoring instruction sent by the APP remote terminal and starting a sound acquisition unit and a sensor unit in the pet collar according to the pet behavior monitoring instruction, wherein the sensor unit comprises a heart rate sensor and a temperature sensor;
the heart rate temperature acquisition module is used for testing the heart rate of the pet by using the heart rate sensor and acquiring the pet temperature value of the temperature sensor;
the health state judging module is used for judging whether the pet heart rate and the pet temperature value meet health conditions or not, if the pet heart rate and the pet temperature value meet the health conditions, the pet behavior monitoring instruction is continuously executed, and if the pet heart rate and the pet temperature value do not meet the health conditions, the sound collecting unit is started;
the audio processing module is used for acquiring the initial audio of the sound acquisition unit and obtaining the frame windowing audio through pre-emphasis and frame windowing processing;
the pet demand judging module is used for carrying out end point monitoring on the frame windowing audio and extracting the characteristic parameters of the frame windowing audio; classifying the frame windowing audio into a pre-constructed file pet sound set based on the characteristic parameters, and judging pet requirements reflected by the frame windowing audio according to classification results;
the pet interactive module is used for synchronizing pet video images acquired by the monitoring equipment to the APP remote terminal and receiving the reward punishment instruction sent by the APP remote terminal, and based on the reward punishment instruction, corresponding reward punishment means are taken to realize pet behavior adjustment and pet interaction.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210809103.7A CN115104548B (en) | 2022-07-11 | 2022-07-11 | Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210809103.7A CN115104548B (en) | 2022-07-11 | 2022-07-11 | Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115104548A true CN115104548A (en) | 2022-09-27 |
| CN115104548B CN115104548B (en) | 2022-12-27 |
Family
ID=83331747
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210809103.7A Active CN115104548B (en) | 2022-07-11 | 2022-07-11 | Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115104548B (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160015005A1 (en) * | 2014-07-16 | 2016-01-21 | Elwha Llc | Remote pet monitoring systems and methods |
| US20170097169A1 (en) * | 2015-10-01 | 2017-04-06 | Plug and Play Robotics, LLC | Smart pet-crate with interactive features |
| CN108157219A (en) * | 2017-12-22 | 2018-06-15 | 深圳市航天华拓科技有限公司 | A kind of pet based on convolutional neural networks stops apparatus and method of barking |
| US20180317875A1 (en) * | 2017-05-03 | 2018-11-08 | Andrey Bakhriddinovich Khayrullaev | System and Method for Telemetrically Monitoring a Target Object |
| CN208708444U (en) * | 2018-08-27 | 2019-04-09 | 华中农业大学 | A kind of oestrus of sow monitoring system based on acoustics |
| CN110335617A (en) * | 2019-05-24 | 2019-10-15 | 国网新疆电力有限公司乌鲁木齐供电公司 | A kind of noise analysis method in substation |
| CN112331231A (en) * | 2020-11-24 | 2021-02-05 | 南京农业大学 | Broiler feed intake detection system based on audio technology |
| CN112401888A (en) * | 2020-11-13 | 2021-02-26 | 深圳创维－Rgb电子有限公司 | Pet companion method, system and computer readable storage medium |
| CN215422223U (en) * | 2021-01-25 | 2022-01-07 | 深圳市前海远为科技有限公司 | Pet snack feeding device |
| CN114667948A (en) * | 2022-02-28 | 2022-06-28 | 宇萌信息科技(苏州)有限公司 | Intelligent pet feeding and accompanying system based on Internet of things |
-
2022
- 2022-07-11 CN CN202210809103.7A patent/CN115104548B/en active Active
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160015005A1 (en) * | 2014-07-16 | 2016-01-21 | Elwha Llc | Remote pet monitoring systems and methods |
| US20170097169A1 (en) * | 2015-10-01 | 2017-04-06 | Plug and Play Robotics, LLC | Smart pet-crate with interactive features |
| US20180317875A1 (en) * | 2017-05-03 | 2018-11-08 | Andrey Bakhriddinovich Khayrullaev | System and Method for Telemetrically Monitoring a Target Object |
| CN108157219A (en) * | 2017-12-22 | 2018-06-15 | 深圳市航天华拓科技有限公司 | A kind of pet based on convolutional neural networks stops apparatus and method of barking |
| CN208708444U (en) * | 2018-08-27 | 2019-04-09 | 华中农业大学 | A kind of oestrus of sow monitoring system based on acoustics |
| CN110335617A (en) * | 2019-05-24 | 2019-10-15 | 国网新疆电力有限公司乌鲁木齐供电公司 | A kind of noise analysis method in substation |
| CN112401888A (en) * | 2020-11-13 | 2021-02-26 | 深圳创维－Rgb电子有限公司 | Pet companion method, system and computer readable storage medium |
| CN112331231A (en) * | 2020-11-24 | 2021-02-05 | 南京农业大学 | Broiler feed intake detection system based on audio technology |
| CN215422223U (en) * | 2021-01-25 | 2022-01-07 | 深圳市前海远为科技有限公司 | Pet snack feeding device |
| CN114667948A (en) * | 2022-02-28 | 2022-06-28 | 宇萌信息科技(苏州)有限公司 | Intelligent pet feeding and accompanying system based on Internet of things |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115104548B (en) | 2022-12-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110123367B (en) | Computer device, heart sound recognition method, model training device, and storage medium | |
| CN108899037A (en) | Animal vocal print feature extracting method, device and electronic equipment | |
| CN109065046A (en) | Method, apparatus, electronic equipment and the computer readable storage medium that voice wakes up | |
| US11373757B2 (en) | Method and system for classifying phonocardiogram signal quality | |
| CN110720946A (en) | Intelligent auscultation system based on deep learning | |
| CN110111815A (en) | Animal anomaly sound monitoring method and device, storage medium, electronic equipment | |
| CN113539294A (en) | Method for collecting and identifying sound of abnormal state of live pig | |
| CN112382302A (en) | Baby cry identification method and terminal equipment | |
| Mahmoudi et al. | Sensor-based system for automatic cough detection and classification | |
| CN110889008B (en) | Music recommendation method and device, computing device and storage medium | |
| CN115104548B (en) | Pet behavior adjustment and human-pet interaction method and device based on multimedia information technology | |
| CN113077821A (en) | Audio quality detection method and device, electronic equipment and storage medium | |
| CN116048282B (en) | Data processing method, system, device, equipment and storage medium | |
| CN112672120B (en) | Projector with voice analysis function and personal health data generation method | |
| KR102573186B1 (en) | Apparatus, method, and recording medium for providing animal sound analysis information | |
| WO2022205400A1 (en) | Voice recognition-based safety alerting method and apparatus, and terminal device | |
| US20200168317A1 (en) | Tool for assisting individuals experiencing auditory hallucinations to differentiate between hallucinations and ambient sounds | |
| TWI839796B (en) | Sound monitoring system | |
| CN115545148A (en) | Body sound auscultation device and body sound data diagnosis method | |
| CN109887526B (en) | Method, device, equipment and storage medium for detecting physiological state of ewe | |
| WO2019023989A1 (en) | Human body physiological parameter test method and test system based on smart terminal | |
| WO2022178934A1 (en) | Health testing method and apparatus, and device and storage medium | |
| CN110136819A (en) | One kind is based on Internet of Things intelligence fetus fetal rhythm monitoring system and monitoring method | |
| US20220108704A1 (en) | Real-time detection and alert of mental and physical abuse and maltreatment in the caregiving environment through audio and the environment parameters | |
| Huq | Differentiation of Dry and Wet Cough Sounds using A Deep Learning Model and Data Augmentation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |