WO2022252803A1

WO2022252803A1 - Screening method, device, storage medium, and program product

Info

Publication number: WO2022252803A1
Application number: PCT/CN2022/085099
Authority: WO
Inventors: 许培达; 李靖
Original assignee: 华为技术有限公司
Priority date: 2021-05-31
Filing date: 2022-04-02
Publication date: 2022-12-08
Also published as: CN115414025A

Abstract

A screening method, a device (100), a storage medium, and a program product (14), which belong to the field of respiratory analysis. The method comprises: detecting a coughing sound; sensing an interaction operation of a user by means of an interaction device (11) (S403); acquiring expectoration information corresponding to the interaction operation (S404); and outputting reminder information according to the expectoration information. A coughing sound is detected to serve as a trigger for obtaining expectoration information, and then assists with the respiratory infection screening according to the expectoration information. In this way, expectoration information can be acquired in a timely manner, thereby providing accurate screening and realizing high operability.

Description

Screening method, equipment, storage medium and program product

This application claims the priority of the Chinese patent with the application number 202110603113.0 filed on May 31, 2021, with the title of "screening method, device, storage medium, and program product" filed with the China Patent Office, the entire contents of which are hereby incorporated by reference In this application.

technical field

The present application relates to the field of breath analysis, in particular to a screening method, equipment, storage medium and program product.

Background technique

Respiratory infection (respiratory tract infection) is a kind of common disease, frequently-occurring disease, disease type is complex, and incidence rate increases year by year, and wherein lower respiratory tract infection is the most serious, and mortality rate is high. New coronary pneumonia is a type of lower respiratory tract infection. Because of its infectivity and harmfulness, the prevention and control of new crown is very important, and respiratory infection screening is of great significance.

Existing screenings for infectious diseases typically require a blood test or a test for another bodily fluid, such as saliva, which can take several hours to perform and typically needs to be performed at a central location where blood analysis equipment is located. Traditional screening techniques are not inherently suitable for individuals to perform screening tests on themselves—requiring caregivers to collect, handle, transport, and analyze samples. There is a great unmet need for better screening procedures, especially during disease outbreaks and epidemics.

Contents of the invention

This application provides a screening method, which is suitable for self-examination by non-professionals, and is timely and universal.

In the first aspect, the present application provides a screening method, the method comprising: detecting coughing sounds, sensing the user's interactive operation through an interactive device, acquiring sputum information corresponding to the interactive operation, and according to the sputum information Output a reminder message.

Among them, cough (cough) is a common symptom of the respiratory tract. It is caused by inflammation, foreign body, physical or chemical stimulation of the trachea, bronchial mucosa or pleura. The door opens and the air in the lungs is ejected, usually with sound, so when the user coughs, the coughing sound can be recorded.

In the embodiment of the present application, the coughing sound is detected as a trigger to obtain sputum information, and then according to the sputum information to assist respiratory infection screening, the sputum information can be obtained in time, providing screening accuracy and strong operability. It would be suitable for a non-professional to perform the test on himself or herself, who would be able to obtain results quickly. Users optionally perform rapid off-line analysis of samples, which will include avoiding cross-contamination of test subjects and protecting healthcare workers from infection, and are suitable for example in the case of screening large populations or repeating tests on a single subject Easy to reuse.

Optionally, the interaction device includes a voice interaction device, and the perceiving the user's interaction operation through the interaction device includes: outputting first voice data through the voice interaction device, where the first voice data is used for communicating with The user interacts; receiving an interactive operation input by the user after the voice interaction device outputs the first voice data, wherein the interactive operation includes voice input.

Optionally, the first voice data includes guidance speeches set according to expectoration information, and the guidance speeches include: general reply guidance speech, verification guidance speech, selective guidance speech and inquiry Guide speech.

Optionally, the interaction device may further include a display device; the display device displays text data and interactive elements converted from the first voice data; then the interactive operation further includes a touch action on the interactive elements operate.

Optionally, the interaction device includes a display device, and the sensing the user's interaction operation through the interaction device includes: presenting an interaction interface through the display device, and the interaction interface includes interactive elements, wherein the interactive elements use Interacting with the user; receiving an interactive operation input by the user on the interactive interface.

Optionally, the interactive element includes a first option set according to expectoration information; then the interactive operation includes a touch operation acting on the first option.

Optionally, the interactive element further includes a picture of sputum and an interactive control associated with the picture of sputum, and the interactive operation includes a touch operation acting on the interactive control.

Optionally, the interactive element includes a viewfinder frame of the picture and a shooting control, and the interactive operation includes a touch operation acting on the shooting control.

Optionally, the sputum expectoration information includes: expectoration frequency, sputum color and sputum viscosity.

Optionally, the outputting reminder information according to the expectoration information includes: acquiring a physiological indicator of the user; and outputting reminder information according to the physiological indicator and the expectoration information.

In the second aspect, the present application also provides an electronic device. The structure of the electronic device includes a processor and a memory, and the memory is used to store and support the electronic device to execute the above-mentioned first aspect and its optional implementation manners provided. A program of the screening method, and storing data involved in implementing the screening method provided in the above first aspect and its optional implementation manners. The processor executes the program stored in the memory to execute the method provided in the aforementioned first aspect and its optional implementation manners. The electronic device may also include a communication bus for establishing a connection between the processor and the memory.

In a third aspect, the present application also provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes the above-mentioned first aspect and its optional implementations The screening method described in the method.

In a fourth aspect, the present application also provides a computer program product, the computer program product includes computer program code, and when the computer program code is executed by a computer, the computer program code can cause the computer to execute the above-mentioned The screening method described in the first aspect and its optional implementation manners.

The technical effects obtained by the above-mentioned second aspect, third aspect and fourth aspect are similar to those obtained by the corresponding technical means in the first aspect, and will not be repeated here.

The beneficial effects brought by the technical solution provided by the application at least include:

By detecting the sound of coughing as a trigger to obtain sputum information, and then assisting respiratory infection screening based on the sputum information, the sputum information can be obtained in a timely manner, providing screening accuracy and strong operability. It would be suitable for a non-professional to perform the test on himself or herself, who would be able to obtain results quickly. Users optionally perform rapid off-line analysis of samples, which will include avoiding cross-contamination of test subjects and protecting healthcare workers from infection, and are suitable for example in the case of screening large populations or repeating tests on a single subject Easy to reuse.

Description of drawings

FIG. 1 is a schematic structural diagram of an electronic device provided in an embodiment of the present application;

FIG. 2 is a schematic structural diagram of a mobile phone provided by an embodiment of the present application;

Fig. 3 is a schematic structural diagram of a smart watch provided by an embodiment of the present application;

Fig. 4 is a schematic flow chart of a screening method provided in the embodiment of the present application;

5A to 5E are schematic diagrams of cough sound recording provided by the embodiment of the present application;

6A to 6F are schematic diagrams of an interactive interface provided by an embodiment of the present application;

7A to 7C are schematic diagrams of another interactive interface provided by the embodiment of the present application;

8A to 8C are schematic diagrams of another interactive interface provided by the embodiment of the present application;

Fig. 9A is a schematic diagram of a prompt message provided by the embodiment of the present application;

FIG. 9B is a schematic diagram of another kind of prompt information provided by the embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

Before explaining the embodiment of the present application in detail, the application scenarios involved in the embodiment of the present application will be introduced first.

Those skilled in the art can appreciate that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed by hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be regarded as exceeding the scope of the present invention.

Please refer to FIG. 1 . FIG. 1 is a schematic structural diagram of an electronic device 100 provided in an embodiment of the present application. The electronic device 100 may include: one or more sound pickup devices 10, one or more interactive devices 11, one or more processors 12, memory 13, one or more application programs (not shown), and one or more A plurality of computer programs 14, the above-mentioned devices may be connected by one or more communication buses.

Wherein, the pickup device 10 is used to collect audio, and it can be a microphone array (Microphone Array) composed of a plurality of microphones. The microphone array physically refers to that a plurality of microphones are arranged in an orderly manner, that is to say, a certain number of acoustic sensors (usually a microphone), a system used to sample and process the spatial characteristics of the sound field.

Among them, the interaction device 11 is used to interact with the user to realize human-computer interaction. The interactive device 11 can be a voice interactive device, such as a home voice assistant, a smart speaker, etc., or a visual interactive device, such as a smart screen, a smart TV, etc., or a smart home appliance such as a smart air conditioner, a smart washing machine, etc. The interactive device used in the home scene may also be the interactive device 11 suitable for working scenes such as factories, enterprises, or hospitals, which is not limited herein. Voice interaction devices can perform voice interaction through voice, and visual interaction devices can interact through an interactive interface.

Wherein the one or more computer programs 14 are stored in the above-mentioned memory 13 and configured to be executed by the one or more processors 12, the one or more computer programs 14 include instructions, and the above-mentioned instructions can be used to perform the following Each step in the embodiment. Wherein, all relevant content of each step involved in the following embodiments may be referred to the functional description of the corresponding physical device, and will not be repeated here. Said instructions constitute a software product, which is loaded into memory 13 . When the instructions are executed, the instructions cause the electronic device 100 to perform screening operations based on cough sounds, and in particular the methods provided in the following embodiments. It can be understood that the programming of the software product is directly based on the method described in the embodiment of the present application.

The above-mentioned electronic device 100 may be a device for screening respiratory infections, and may be installed in electronic devices (such as mobile phones, smart watches, wristbands, tablet PCs, desktop computers, laptop computers, etc.), or installed in In medical devices used in dedicated medical institutions. In addition, equipment for screening for respiratory infections can be manufactured as a stand-alone hardware device, such as a wearable device worn on a subject, and examples of wearable devices include wristwatch type wearable devices, bracelet type wearable devices, wrist A belt-type wearable device, a ring-type wearable device, a glasses-type wearable device, a headband-type wearable device, etc., but the wearable device is not limited thereto.

Taking the electronic device as a mobile phone as an example, please refer to FIG. 2 . FIG. 2 is a schematic structural diagram of a mobile phone provided by an embodiment of the present application.

The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (universal serial bus, USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, and an antenna 2 , mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, earphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and A subscriber identification module (subscriber identification module, SIM) card interface 195 and the like. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, bone conduction sensor 180M, etc.

It can be understood that, the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine certain components, or separate certain components, or arrange different components. The illustrated components can be realized in hardware, software or a combination of software and hardware.

The processor 110 may include one or more processing units, for example: the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processing unit (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural network processor (neural-network processing unit, NPU), etc. Wherein, different processing units may be independent devices, or may be integrated in one or more processors.

The controller can generate an operation control signal according to the instruction opcode and timing signal, and complete the control of fetching and executing the instruction.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is a cache memory. The memory may hold instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated access is avoided, and the waiting time of the processor 110 is reduced, thus improving the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. The interface may include an integrated circuit (inter-integrated circuit, I2C) interface, an integrated circuit built-in audio (inter-integrated circuit sound, I2S) interface, a pulse code modulation (pulse code modulation, PCM) interface, a universal asynchronous transmitter (universal asynchronous receiver/transmitter, UART) interface, mobile industry processor interface (mobile industry processor interface, MIPI), general-purpose input and output (general-purpose input/output, GPIO) interface, subscriber identity module (subscriber identity module, SIM) interface, and /or universal serial bus (universal serial bus, USB) interface, etc.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (serial data line, SDA) and a serial clock line (derail clock line, SCL). In some embodiments, processor 110 may include multiple sets of I2C buses. The processor 110 can be respectively coupled to the touch sensor 180K, the charger, the flashlight, the camera 193 and the like through different I2C bus interfaces. For example, the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 and the touch sensor 180K communicate through the I2C bus interface to realize the touch function of the electronic device 100 .

The I2S interface can be used for audio communication. In some embodiments, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 through an I2S bus to implement communication between the processor 110 and the audio module 170 . In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the I2S interface, so as to realize the function of answering calls through the Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing and encoding the analog signal. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled through a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to realize the function of answering calls through the Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, a UART interface is generally used to connect the processor 110 and the wireless communication module 160 . For example: the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to realize the Bluetooth function. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 through the UART interface, so as to realize the function of playing music through the Bluetooth headset.

The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193 . MIPI interface includes camera serial interface (camera serial interface, CSI), display serial interface (display serial interface, DSI), etc. In some embodiments, the processor 110 communicates with the camera 193 through the CSI interface to realize the shooting function of the electronic device 100 . The processor 110 communicates with the display screen 194 through the DSI interface to realize the display function of the electronic device 100 .

The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193 , the display screen 194 , the wireless communication module 160 , the audio module 170 , the sensor module 180 and so on. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, MIPI interface, etc.

The USB interface 130 is an interface conforming to the USB standard specification, specifically, it can be a Mini USB interface, a Micro USB interface, a USB Type C interface, and the like. The USB interface 130 can be used to connect a charger to charge the electronic device 100 , and can also be used to transmit data between the electronic device 100 and peripheral devices. It can also be used to connect headphones and play audio through them. This interface can also be used to connect other electronic devices, such as AR devices.

It can be understood that the interface connection relationship between the modules shown in the embodiment of the present application is only a schematic illustration, and does not constitute a structural limitation of the electronic device 100 . In other embodiments of the present application, the electronic device 100 may also adopt different interface connection manners in the foregoing embodiments, or a combination of multiple interface connection manners.

The charging management module 140 is configured to receive a charging input from a charger. Wherein, the charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 can receive charging input from the wired charger through the USB interface 130 . In some wireless charging embodiments, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the electronic device 100 . While the charging management module 140 is charging the battery 142 , it can also provide power for electronic devices through the power management module 141 .

The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives the input from the battery 142 and/or the charging management module 140 to provide power for the processor 110 , the internal memory 121 , the display screen 194 , the camera 193 , and the wireless communication module 160 . The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle times, and battery health status (leakage, impedance). In some other embodiments, the power management module 141 may also be disposed in the processor 110 . In some other embodiments, the power management module 141 and the charging management module 140 may also be set in the same device.

The wireless communication function of the electronic device 100 can be realized by the antenna 1 , the antenna 2 , the mobile communication module 150 , the wireless communication module 160 , a modem processor, a baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: Antenna 1 can be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 can provide wireless communication solutions including 2G/3G/4G/5G applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 150 can receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and send them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signals modulated by the modem processor, and convert them into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least part of the functional modules of the mobile communication module 150 may be set in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 and at least part of the modules of the processor 110 may be set in the same device.

A modem processor may include a modulator and a demodulator. Wherein, the modulator is used for modulating the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator sends the demodulated low-frequency baseband signal to the baseband processor for processing. The low-frequency baseband signal is passed to the application processor after being processed by the baseband processor. The application processor outputs sound signals through audio equipment (not limited to speaker 170A, receiver 170B, etc.), or displays images or videos through display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In some other embodiments, the modem processor may be independent from the processor 110, and be set in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless Fidelity, Wi-Fi) network), bluetooth (bluetooth, BT), global navigation satellite, etc. applied on the electronic device 100. System (global navigation satellite system, GNSS), frequency modulation (frequency modulation, FM), near field communication technology (near field communication, NFC), infrared technology (infrared, IR) and other wireless communication solutions. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , frequency-modulate it, amplify it, and convert it into electromagnetic waves through the antenna 2 for radiation.

In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (general packet radio service, GPRS), code division multiple access (code division multiple access, CDMA), broadband Code division multiple access (wideband code division multiple access, WCDMA), time division code division multiple access (time-division code division multiple access, TD-SCDMA), long term evolution (long term evolution, LTE), BT, GNSS, WLAN, NFC , FM, and/or IR techniques, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (global navigation satellite system, GLONASS), a Beidou navigation satellite system (beidou navigation satellite system, BDS), a quasi-zenith satellite system (quasi -zenith satellite system (QZSS) and/or satellite based augmentation systems (SBAS).

The electronic device 100 realizes the display function through the GPU, the display screen 194 , and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

The display screen 194 is used to display images, videos and the like. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active matrix organic light emitting diode or an active matrix organic light emitting diode (active-matrix organic light emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light emitting diodes (quantum dot light emitting diodes, QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

The electronic device 100 can realize the shooting function through the ISP, the camera 193 , the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used for processing the data fed back by the camera 193 . For example, when taking a picture, open the shutter, the light is transmitted to the photosensitive element of the camera through the lens, and the light signal is converted into an electrical signal, and the photosensitive element of the camera transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin color. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be located in the camera 193 .

Camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects it to the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

Digital signal processors are used to process digital signals. In addition to digital image signals, they can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the energy of the frequency point.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, for example: moving picture experts group (moving picture experts group, MPEG) 1, MPEG2, MPEG3, MPEG4 and so on.

The NPU is a neural-network (NN) computing processor. By referring to the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process input information and continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as image recognition, face recognition, speech recognition, text understanding, and the like.

The internal memory 121 may include one or more random access memories (random access memory, RAM) and one or more non-volatile memories (non-volatile memory, NVM).

Random access memory can include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (synchronous dynamic random access memory, SDRAM), double data rate synchronous Dynamic random access memory (double data rate synchronous dynamic random access memory, DDR SDRAM, such as the fifth generation DDR SDRAM is generally called DDR5SDRAM), etc.;

Non-volatile memory may include magnetic disk storage devices, flash memory (flash memory).

According to the operating principle, flash memory can include NOR FLASH, NAND FLASH, 3D NAND FLASH, etc. According to the potential order of storage cells, it can include single-level storage cells (single-level cell, SLC), multi-level storage cells (multi-level cell, MLC), triple-level cell (TLC), quad-level cell (QLC), etc., can include universal flash storage (English: universal flash storage, UFS) according to storage specifications , embedded multimedia memory card (embedded multi media Card, eMMC), etc.

The random access memory can be directly read and written by the processor 110, and can be used to store executable programs (such as machine instructions) of an operating system or other running programs, and can also be used to store data of users and application programs.

The non-volatile memory can also store executable programs and data of users and application programs, etc., and can be loaded into the random access memory in advance for the processor 110 to directly read and write.

The external memory interface 120 can be used to connect an external non-volatile memory, so as to expand the storage capacity of the electronic device 100 . The external non-volatile memory communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music and video are stored in an external non-volatile memory.

The electronic device 100 can implement audio functions through the audio module 170 , the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playback, recording, etc.

The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In some embodiments, the audio module 170 may be set in the processor 110 , or some functional modules of the audio module 170 may be set in the processor 110 .

Speaker 170A, also referred to as a "horn", is used to convert audio electrical signals into sound signals. Electronic device 100 can listen to music through speaker 170A, or listen to hands-free calls.

Receiver 170B, also called "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or a voice message, the receiver 170B can be placed close to the human ear to receive the voice.

The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can put his mouth close to the microphone 170C to make a sound, and input the sound signal to the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 may be provided with two microphones 170C, which may also implement a noise reduction function in addition to collecting sound signals. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and realize directional recording functions, etc.

The earphone interface 170D is used for connecting wired earphones. The earphone interface 170D can be a USB interface 130, or a 3.5mm open mobile terminal platform (OMTP) standard interface, or a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, pressure sensor 180A may be disposed on display screen 194 . There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may be comprised of at least two parallel plates with conductive material. When a force is applied to the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 may also calculate the touched position according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities may correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view short messages is executed. When a touch operation whose intensity is greater than or equal to the first pressure threshold acts on the icon of the short message application, the instruction of creating a new short message is executed.

The gyro sensor 180B can be used to determine the motion posture of the electronic device 100 . In some embodiments, the angular velocity of the electronic device 100 around three axes (ie, x, y and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B can be used for image stabilization. Exemplarily, when the shutter is pressed, the gyro sensor 180B detects the shaking angle of the electronic device 100, calculates the distance that the lens module needs to compensate according to the angle, and allows the lens to counteract the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.

The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude based on the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 may use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a clamshell machine, the electronic device 100 can detect opening and closing of the clamshell according to the magnetic sensor 180D. Furthermore, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, features such as automatic unlocking of the flip cover are set.

The acceleration sensor 180E can detect the acceleration of the electronic device 100 in various directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

The distance sensor 180F is used to measure the distance. The electronic device 100 may measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 may use the distance sensor 180F for distance measurement to achieve fast focusing.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The light emitting diodes may be infrared light emitting diodes. The electronic device 100 emits infrared light through the light emitting diode. Electronic device 100 uses photodiodes to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it may be determined that there is an object near the electronic device 100 . When insufficient reflected light is detected, the electronic device 100 may determine that there is no object near the electronic device 100 . The electronic device 100 can use the proximity light sensor 180G to detect that the user is holding the electronic device 100 close to the ear to make a call, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode, automatic unlock and lock screen in pocket mode.

The ambient light sensor 180L is used for sensing ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in the pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, take pictures with fingerprints, answer incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to implement a temperature treatment strategy. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the electronic device 100 may reduce the performance of the processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to prevent the electronic device 100 from being shut down abnormally due to the low temperature. In some other embodiments, when the temperature is lower than another threshold, the electronic device 100 boosts the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.

The touch sensor 180K is also called "touch device". The touch sensor 180K can be disposed on the display screen 194 , and the touch screen includes the touch sensor 180K and the display screen 194 , also called “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194 . In some other embodiments, the touch sensor 180K may also be disposed on the surface of the electronic device 100, which is different from the position of the display screen 194.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the human pulse and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined into a bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vibrating bone mass of the vocal part acquired by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

In the embodiment of the present application, some sensors can be added to the electronic device 100, and physiological data can be obtained through the sensors, which can then be used to measure the user's physiological indicators, including heart rate, body temperature, blood oxygen and respiration rate, etc.

In one possible implementation manner, the electronic device 100 includes a blood oxygen module, and the blood oxygen module includes two red light sensors, at least two infrared sensors, a green light sensor, and at least two red light sensors for At least two red light signals are acquired; at least two infrared sensors are used to acquire at least two infrared signals; a green light sensor is used to acquire green light signals. The processor is used for determining the component signals of the red direct current data and the red alternating current signal according to at least two red light signals; and determining the infrared direct current data and the component signals of the infrared alternating current signal according to at least two infrared signals. The component signals include arterial signals. The processor is also used to determine the red light AC data according to the component signal of the red light AC signal and the green light signal; and determine the infrared light AC data according to the component signal of the infrared AC signal and the green light signal. The processor is also configured to determine blood oxygen saturation based on the red light direct current data, the red light alternating current data, the infrared direct current data, and the infrared alternating current data.

In one possible implementation manner, the electronic device 100 includes a heart rate module, which is used to collect the data of the user's blood vessels changing with the pulse when testing the heart rate, for the processor to calculate the heart rate value through the heart rate algorithm. The heart rate module may include a control chip for implementing a heart rate algorithm, a PPG sensor, and an acceleration sensor.

In one possible implementation manner, the electronic device 100 is configured with a photoplethysmograph (Photoplethysmograph, PPG) to measure heart rate, and an electrocardiogram (Electrocardiogram, ECG) to measure ECG.

In one possible implementation manner, the electronic device 100 includes a temperature sensor, and the user's body temperature is measured by the temperature sensor.

In one of the possible implementation manners, the electronic device 100 may use a standard non-contact microphone to generate an original signal representing the airflow sound of breathing, and analyze the original signal to determine one or more respiratory parameters of the first subgroup, and A second subset of one or more estimated respiratory parameters is derived that is typically not directly detectable in the raw signal. Among them: the first subgroup of parameters includes active breathing time (duration of active exhalation) and respiratory period (time between successive breaths), the second subgroup of parameters includes inspiratory time, and then the respiratory rate can be calculated, that is, breathing frequency.

The keys 190 include a power key, a volume key and the like. The key 190 may be a mechanical key. It can also be a touch button. The electronic device 100 can receive key input and generate key signal input related to user settings and function control of the electronic device 100 .

The motor 191 can generate a vibrating reminder. The motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations applied to different applications (such as taking pictures, playing audio, etc.) may correspond to different vibration feedback effects. The motor 191 may also correspond to different vibration feedback effects for touch operations acting on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, and can be used to indicate the charging status, the change of the battery capacity, and can also be used to indicate messages, missed calls, notifications and the like.

The SIM card interface 195 is used for connecting a SIM card. The SIM card can be connected and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card etc. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards may be the same or different. The SIM card interface 195 is also compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calling and data communication. In some embodiments, the electronic device 100 adopts an eSIM, that is, an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

Taking the electronic device as an example of a smart watch, please refer to FIG. 3 , which is a schematic structural diagram of a smart watch provided by an embodiment of the present application.

As shown in FIG. 3 , the smart watch may include: a processor 201 , a memory 202 , a sensor 203 , at least one receiver 204 , at least one microphone 205 , a power source 206 and a Bluetooth device 207 .

Wherein, the memory 202 may be used to store application program codes, such as application program codes for pairing the smart watch with the above-mentioned second electronic device. The processor 201 can control the application program code that executes the screening method of the embodiment of the present application, so as to realize the function of the smart watch in the embodiment of the present application to perform respiratory infection screening.

A Bluetooth address for uniquely identifying the smart watch may also be stored in the memory 202 . In addition, the memory 202 may also store connection data of electronic devices that have been successfully paired with the smart watch before. For example, the connection data may be the Bluetooth address of the electronic device that has been successfully paired with the smart watch. Based on the connection data, the smart watch can be automatically paired with the electronic device without having to configure the connection with it, such as performing legality verification. The above bluetooth address may be a media access control (media access control, MAC) address.

The sensor 203 may include a distance sensor, a proximity light sensor, a gravity sensor and/or a gyroscope and the like. The processor 201 of the smart watch can determine whether the watch is worn by the user through the sensor 203, and can also use the sensor 203 to determine some actions of the user when the watch is worn, such as raising the wrist. For example, the processor 201 of the smart watch can use the proximity light sensor to detect whether there is an object near the smart watch, so as to determine whether the smart watch is worn by the user. When it is determined that the smart watch is worn, the processor 201 of the smart watch may turn on the receiver 204 . In some embodiments, the smart watch may also include bone conduction sensors, combined with bone conduction earphones. The bone conduction sensor can acquire the vibration signal of the vibrating bone block of the vocal part, and the processor 201 analyzes the voice signal to realize the control function corresponding to the voice signal. In some other embodiments, the smart watch may further include a touch sensor or a pressure sensor, which are respectively used to detect a user's touch operation and press operation. In other embodiments, the smart watch may further include a fingerprint sensor for detecting the user's fingerprint, identifying the user's identity, and the like. In other embodiments, the smart watch may further include an ambient light sensor, and the processor 201 of the smart watch may adaptively adjust some parameters, such as volume, according to the brightness of the ambient light sensed by the ambient light sensor.

The Bluetooth device 207 is used to establish a Bluetooth connection with other electronic devices (such as the second electronic device), so that short-distance data exchange can be performed between the first electronic device and other electronic devices.

The receiver 204, also called "earpiece", can be used to convert audio electrical signals into audio signals and play them. For example, when the smart watch is used as the audio output device of the above-mentioned second electronic device, the receiver 204 may convert the received audio electrical signal into a sound signal and play it.

The microphone 205, which may also be referred to as "microphone" or "microphone", is used to convert sound signals into audio electrical signals. For example, when a smart watch is used as the audio input device of the above-mentioned second electronic device, when the user speaks (such as talking or sending a voice message), the microphone 205 can collect the user's voice signal and convert it into an audio electrical signal.

The power supply 206 can be used to supply power to various components contained in the smart watch. In some embodiments, the power source 206 may be a battery, such as a rechargeable battery.

In the embodiment of this application, some sensors can be added to the smart watch, and physiological data can be obtained through the sensors, which can then be used to measure the user's physiological indicators, including heart rate, body temperature, blood oxygen and breathing rate, etc.

It can be understood that the structure shown in this embodiment does not constitute a specific limitation on the smart watch. It may have more or fewer components than shown in FIG. 2 , may combine two or more components, or may have a different configuration of components. For example, the smart watch can also include components such as an indicator light (which can indicate the state of the battery, etc.), a dust-proof net (which can be used in conjunction with the earpiece). The various components shown in Figure 2 may be implemented in hardware, software, or a combination of hardware and software including one or more signal processing or application specific integrated circuits.

Next, the screening method provided in the embodiment of the present application will be introduced.

The flow chart is used in this application to describe the operations performed by the device according to the embodiment of this application. It should be understood that the preceding or following operations are not necessarily performed in an exact order. Instead, various steps may be processed in reverse order or concurrently, as desired. At the same time, other operations can be added to these procedures, or a certain step or steps can be removed from these procedures.

Please refer to FIG. 4, which is a schematic flowchart of the screening method provided in the embodiment of the present application. The screening method can be performed by an electronic device, and the method includes the following steps:

Step S401: Record coughing sound.

In this embodiment of the application, taking electronic devices including smart watches and/or mobile phones as an example, the sound pickup device (such as a microphone) on the smart watch or mobile phone can collect sound signals from the external environment of the smart watch or mobile phone, and the collected sound The sound characteristics of the signal are analyzed and the cough sound is recorded. The environmental sound signals can be collected through the microphones arranged in the electronic equipment respectively, and the environmental sound signals can be processed through the processor arranged in the electronic equipment; The ambient sound signal is processed. This application does not specifically limit it. Wherein, to process the environmental sound signal, a speech processing algorithm may be used to identify the collected environmental sound, recognize the cough sound in it, and extract the cough sound in it, which will not be described in detail here.

In the embodiment of the present application, the screening method in the embodiment of the present application can be realized as an APP or a certain function in the APP. For example, the screening method in the embodiment of the present application can be realized as the respiratory infection screening function in the sports health APP , or implemented as a respiratory infection app, which is not specifically limited in this application.

A sports health APP is installed on a smart watch or a mobile phone, and the sports health application APP can provide respiratory infection screening function. Or a respiratory infection screening APP is installed on a smart watch or a mobile phone, and the respiratory infection screening APP can provide a respiratory infection screening function. In practical applications, according to the spontaneity of respiratory infection function usage, it can be divided into active screening scenarios and passive screening scenarios. The active screening scenario allows users to actively use the respiratory infection screening function in the respiratory infection screening APP to perform respiratory infection screening, that is, users can initiate screening at any time to understand their own respiratory infection risk. The passive screening scenario is that after the user authorizes, the respiratory infection screening app automatically invokes its own respiratory infection screening function for respiratory infection screening.

The following describes the recording process of the cough sound in the active screening scenario.

Please refer to FIG. 5A to FIG. 5E together. The user opens the sports health APP or the respiratory infection screening APP and enters the respiratory infection screening interface 50 , as shown in FIG. 5A . A data frame 501 and a start detection button 502 are presented on the respiratory infection screening interface 50 . The data frame 501 displays the data measured in the past. If there is no measurement data, it will be displayed as no data today. Click the start detection button 502 shown in Figure 5A, after clicking the start detection button 502, if the user is using it for the first time, a guide interface 51 will appear, as shown in Figure 5B. The guidance interface 51 may include a measurement tutorial 511 , a measurement technique 512 and a start detection button 502 , and the measurement tutorial 511 includes: first step: recording a coughing sound. Step 2: Collect physiological indicators. Measurement skills 512 include: the measurement process takes about 1 minute and 30 seconds, and the entire measurement process can refer to the measurement tutorial. 1. Record cough sound: After clicking to start detection, cough 3 times into the microphone, and please keep the surrounding environment relatively quiet. 2. Wear the wearable device and keep quiet to collect physiological data for 1 minute. After the user clicks the start detection button 502 shown in Figure 5B, or the user is not using it for the first time, after clicking the start detection button 502 shown in Figure 5A, a test interface 52 is presented, as shown in Figure 5C, the test interface 52 includes a recording button 521. After the user clicks the recording button 521, a recording interface 53 is presented. As shown in FIG. 5D, the recording duration is displayed on the recording interface 53. In response to the user clicking the recording button 521, the sound pickup device of the electronic device starts to collect sound. At this time, the user can compulsively cough (not a natural cough), usually 3-5 consecutive coughs, and the whole process takes about 20 seconds. In order to ensure that the recorded sound is a cough sound, cough sound recognition will be performed. If it is not a cough sound or the quality of the cough sound is poor, the electronic device can prompt the user to re-record the cough sound. When the cough sound is recorded, the user can be prompted to let the user There is an angular deviation between the mouth and the sound pickup device, so that the sound will not be directly aimed at the sound pickup device, so as to avoid excessive audio amplitude and distortion, exceeding the upper limit of the sound pickup range of the sound pickup device. After the recording is finished, the electronic device records the cough sound collected by the sound pickup device, and executes the following steps S402 to S406 when the cough sound is detected.

In one possible implementation manner, other physiological data of the user may also be collected after recording the coughing sound or while recording the coughing sound. Exemplarily, after the recording ends, as shown in FIG. 5E , a physiological data collection interface 54 can also be presented, and the physiological data collection interface 54 prompts the user to wear a wearable device (such as a smart watch), keep quiet, and sit comfortably to collect The user's physiological data, for example, collect physiological data such as pulse wave and body temperature through the wrist smart watch. The smart watch is equipped with sensors to obtain physiological data through the sensor, and obtain the user's physiological indicators according to the collected physiological data.

In one possible implementation, the electronic device in the embodiment of the present application may include a plurality of different electronic devices, such as a smart watch and a mobile phone respectively, and the cough sound is recorded through the mobile phone, and the physiological data of the user is collected through the smart watch.

In one possible implementation manner, the electronic device may be the same device, such as a mobile phone, which collects and records the user's coughing sound through the mobile phone, and collects the user's physiological data through the mobile phone to obtain the user's physiological indicators. Or for a smart watch, collect and record the user's coughing sound through the smart watch, and collect the user's physiological data through the smart watch to obtain the user's physiological indicators.

The following describes the recording process of cough sounds in passive screening scenarios.

Install the sports health APP or respiratory infection screening APP on the smart watch or mobile phone. The functions of the sports health APP or respiratory infection screening APP include automatic screening (background screening, non-inductive screening), that is, after the user authorizes and agrees, the smart The sound pickup device of the watch or mobile phone can continuously record the user's voice data, and judge whether the recorded sound is a cough sound according to the sound data within a certain period of time. If it is a cough sound, record the cough sound. The cough sound recorded here It is a cough that occurs naturally on the user's own physiology, while the coughing sound of the user in the automatic screening scenario is mostly a cough that occurs under human control. Exemplarily, the user authorizes the sports health APP or the respiratory infection screening APP, and the sports health APP or the respiratory infection screening APP can record the sound of the external environment in real time, and recognize the sound of the external environment, and recognize the coughing sound in the detection. When a cough sound is heard, the following steps S402 to S406 are performed.

In the embodiment of this application, if the user can predict that he is about to cough, he can independently use the respiratory infection screening function of the Sports Health APP or the respiratory infection screening APP, and refer to the guidance shown in Figure 5A to Figure 5D to cough to The electronic device is made to record the cough sound, so that when the recorded sound is judged to be a cough sound, the respiratory infection screening function of the APP is triggered to execute the screening method of the embodiment of the present application. Or the user has authorized the APP to allow the APP to collect and record environmental sounds in real time. If the user wants to test and check his own respiratory infection, he can artificially cough so that the electronic device can record the coughing sound, so that when the recorded sound is judged to be a coughing sound , trigger the respiratory infection screening function of the APP, and execute the screening method of the embodiment of the present application.

Step S402: Process the cough sound, and determine whether the cough sound is a wet cough.

In the embodiment of the present application, the recorded cough sound can be input into the cough sound recognition model, and the cough sound recognition model can judge whether it is a dry cough or a wet cough. The cough sound of a person who coughs but has no sputum, the cough sound of a person who has a cough and sputum, and then trains a deep learning model or a shallow learning model based on the collected audio data. The trained deep learning model or shallow learning model is Cough sound recognition model.

In the embodiment of the present application, step S402 can be omitted, that is, step S403 can be executed after the cough sound is detected, and there is no need to judge whether the cough sound is a wet cough, and step S403 can be executed after the cough sound is detected, that is, to detect The sound of coughing is used as a triggering event for respiratory infection screening, and the screening method of the embodiment of the present application is executed when the sound of coughing is detected.

If it is judged in step S402 that the cough sound is a wet cough, then step S403 is executed. If it is judged in step S402 that the cough sound is not a wet cough, then step S406 is executed.

Step S403: Perceiving the user's interactive operation through the interactive device.

In the embodiment of the present application, the detection of a cough sound as a wet cough may be used as a trigger event for respiratory infection screening, and the screening method of the embodiment of the present application is performed when a wet cough sound is detected.

In the embodiment of the present application, the interactive device is communicatively connected to the electronic device, and the interactive device may be a part of the electronic device. Interactive devices can be voice interactive devices, such as home voice assistants, smart speakers, etc., or visual interactive devices, such as smart screens, smart TVs or touch screens, etc., or smart home appliances such as smart air conditioners, smart washing machines, etc. In addition to the aforementioned interactive devices applicable to home scenarios, they may also be interactive devices applicable to working scenarios such as factories, enterprises, or hospitals, without any limitation herein.

In this embodiment of the present application, in the first case, the interaction device may only include a voice interaction device. In the second case, the interaction device may include a voice interaction device and a display device. In a third scenario, the interaction device may only comprise a display device.

In the first case, for example, the interaction device may include a voice interaction device through which the voice interaction with the user is performed, that is, the voice interaction device may include a voice pickup device, a voice playback device, and a processor, and the voice pickup device may Collect environmental sounds to obtain coughing sounds. The sound pickup device can also collect the user's voice when interacting with the user. The processor processes the user's voice collected by the sound pickup device, and outputs the corresponding For the first voice data, play the first voice data through the voice playback device.

In this embodiment of the present application, the first voice data may be output through the voice interaction device, wherein the first voice data is used to interact with the user, and an interaction operation input by the user after the voice interaction device outputs the first voice data is received, wherein Interactions include voice input. Wherein, the first voice data includes guidance speeches set according to the expectoration information, and the guidance speeches include: general reply guidance speech, verification guidance speech, selection guidance speech and inquiry guidance speech.

In the embodiment of the present application, the guidance speech technique set according to the sputum information, that is, the guidance speech technique is used to guide the user to obtain sputum information from the user, including whether the cough contains sputum, whether the sputum is transparent, whether the sputum is Is it sticky etc.

Among them, the general reply guidance speech is used to indicate that the behavior of the electronic device only returns a one-sentence guidance, without multiple rounds of dialogue interaction. The general reply guidance speech can be, for example, "I didn't hear clearly just now, you can talk to me again." One more time?", the interaction corresponding to this behavior type is applicable to situations where the user's intention cannot be recognized based on the user's voice.

Among them, the verification-type guidance behavior speech is used to indicate that the electronic device asks the user for a resource selection and expects the user to return "yes" or "no" in the next round of dialogue. The guidance speech can be, for example, "I didn't hear clearly, you Did you mean you didn’t cough?”, the interaction corresponding to this behavior type is suitable for the situation where the user’s intention is inferred based on the user’s voice, and there are few options for feedback to the user.

Among them, the selective guidance behavior is used to indicate that the electronic device feeds back multiple choices to the user and expects the user to select one of the options in the next round of dialogue. The guidance speech can be, for example, "What color is sputum? A: transparent, B : Rust color?”, the interaction corresponding to this behavior type is applicable to situations where the user’s intent is estimated based on the user’s voice, and there are many options for feedback to the user.

Among them, the inquiry-type guidance behavior is used to indicate that the electronic device returns to the user a guidance to inquire about the slot and expect the user to answer, wherein the slot refers to a keyword related to the user's intention in the user's voice, and the guidance speech can be, for example, "Are you What song do you want to listen to by Jay Chou?", the slot is "Jay Chou". The interaction corresponding to this behavior type is applicable to the situation where the user's intention can be determined based on the user's voice, and the user's needs need to be further clarified.

Realize audio interaction through the voice interaction device, specifically: when the electronic device detects a cough sound, or judges that the cough sound is a wet cough, the processor in the voice interaction device controls the voice playback device to output the first voice data "XXX, hello, Did you cough just now?" Then, the voice input of the user is collected by the sound pickup device, and the collected voice input of the user is processed, and if it is determined that the user answers "no", then the process ends. If it is judged that the user answers "Yes", then continue, and the processor controls the voice playback device to output the first voice data "Do you have phlegm in your cough?". Then the user's voice input is collected by the sound pickup device, and the collected user's voice input is processed. If it is judged that the user answers "No", it is considered to be finished. If it is judged that the user answers "Yes", then continue, and the processor controls the voice playback device to output the first voice data "Is the sputum thick?". Then, the user's voice input is collected by the sound pickup device, the collected user voice input is processed, and the information input by the user is recorded. If it is judged that the user answers "yes", then "sputum is thick" is recorded. If it is judged that the user answers "no", then record "sputum is not viscous". After receiving the voice input by the user, control the voice playback device to output the first voice data "What color is the sputum?" Then collect the user's voice input through the voice pickup device, process the collected user voice input, and record the user's input. For information, users can input transparent, rust, brick red, golden yellow pus or yellow green pus, etc. Recognize the speech entered by the user and record the color entered by the user. In one possible implementation, the voice playback device is controlled to output the first voice data "What color is the sputum? A: transparent, B: rust color, C: brick red, D: golden yellow pus sputum, E: yellow Green pus sputum. Then determine the user's choice, if the user chooses A, then the recorded sputum is transparent. After receiving the voice input by the user, control the voice playback device to output the first voice data "Have you coughed up a lot of sputum recently? ", and then collect the user's voice input through the sound pickup device, process the collected user voice input, and record the information input by the user. If it is judged that the user answers "a lot", then record "a lot of phlegm". If it is judged that the user answers "Less", then record "less sputum". Finally, control the voice playback device to output the first voice data "Thank you for your feedback".

In the second case, the interaction device may include a voice interaction device and a display device, and the display device displays text data converted from the first voice data, and may also display voice waveforms. That is, the first voice data is played and output with the voice playback device, and the text data converted from the first voice data is displayed and output through the display device, and the first voice data can be output while the text data is output. The display device may display text data converted from the first voice data and interactive elements, and the interactive operation between the user and the electronic device may be a touch operation acting on the interactive elements. That is, when the interaction device includes a voice interaction device and a display device, the user may perform voice interaction through the voice interaction device, or perform interaction on the display device.

For example, please refer to FIG. 6A together. When the voice playback device outputs the first voice data "XXX, hello, did you cough just now?", the display device displays the converted text data of the first voice data "XXX , hello, did you cough just now?", the display device also displays interactive elements, which can be the answer options for the above questions, such as the option "yes" and the option "no", to detect the user's interactive operation, such as detecting Whether the user inputs voice, or detects whether the user inputs an operation on the display device, such as a touch operation on the display device (click, long press, etc.). Detect that the user clicks the "No" option, then end. Detect that the user clicks "Yes", and record a cough. Then the voice playback device outputs the first voice data "Do you have phlegm in your cough?" Please refer to Figure 6B together, the text data converted from the first voice data "Do you have phlegm in your cough?" and options are displayed on the display device "Yes" and option "No". Detect that the user clicks the "No" option, then end. It is detected that the user clicks "Yes", and the cough with sputum is recorded. The voice playback device outputs the first voice data "Is the sputum sticky?" Please refer to Figure 6C together, the text data converted from the first voice data "Is the sputum sticky?" and the option "Yes" are displayed on the display device and option "No". If it is detected that the user clicks the "No" option, then "sputum is not viscous" is recorded. When it is detected that the user clicks "Yes", it will record "Sputum is thick", and the voice playback device outputs the first voice data "What color is the sputum? A: Transparent, B: Rust color, C: Brick red, D: Gold Yellow purulent sputum, E: yellow-green purulent sputum.", please refer to Figure 6D together, the display device displays the text data after the conversion of the first voice data "What color is the sputum? A: transparent, B: rusty, C: Brick red, D: Golden yellow purulent sputum, E: Yellow green purulent sputum." It is detected that the user chooses B, and the recorded sputum is rust-colored. It is detected that the user clicks option B, and the sputum is recorded as rust-colored. The voice playback device outputs the first voice data "Have you coughed up a lot recently?" Please also refer to Figure 6E, the display device displays the converted text data of the first voice data "Have you coughed up a lot recently?" and options A: many, B: no more, no less, C: not much, if it is detected that the user selects the option "many", it will record "a lot of sputum". If it is detected that the user selects the option "less", then "low sputum volume" is recorded. Finally, the voice playback device is controlled to output "Thank you for your feedback". Please also refer to Figure 6F, which shows "Thank you for your feedback" on the display device.

In the third case, the interaction device only includes a display device, and the sensing the user's interaction operation through the interaction device includes: presenting an interaction interface through the display device, and the interaction interface includes interactive elements, wherein the The interactive element is used for interacting with the user; receiving the interactive operation input by the user on the interactive interface (such as click, long press, input text). That is, the user can interact with the electronic device through the interactive interface. The interactive element includes a first option set according to the expectoration information; then the interactive operation includes a touch operation acting on the first option.

Question information can be presented separately through multiple interactive interfaces, or all questions can be presented on the interactive interface at one time, allowing users to pull down and drag pages to answer questions.

As an example, please refer to FIG. 7A together. Question 1 is presented on the interactive interface 70: "Which of the following is the most consistent with the sputum information of your cough?", and the first option "A: No sputum, B: transparent, C: rust color, D: brick red, E: golden yellow purulent sputum, F: yellow green purulent sputum, G: other input". Question 2: "Have you coughed up a lot of sputum recently?", and the first option "A: A lot, B: Not too much, C: Not much" is presented. The user's interactive operation on the first option on the interactive interface 70 is monitored. For question 1, if the user clicks on option A, it will be recorded that this cough has no phlegm. If the user clicks option B, it will be recorded that the cough sputum is transparent. If the user clicks option C, it will be recorded that the cough sputum is rust-colored. If the user clicks on option D, it will be recorded that the cough sputum is brick red. If the user clicks option E, it is recorded that the cough sputum is golden yellow purulent sputum. If the user clicks option F, it will be recorded that the cough sputum is yellow-green purulent sputum. If the user clicks option G, the user can directly input relevant information in text, such as inputting "yellow", or uploading a picture of the coughing sputum.

In one of the possible implementations, please refer to FIG. 7B together. The difference between FIG. 7B and FIG. 7A is that the question "Did you cough just now?" is also presented on the interactive interface 70, and the first option "Yes" and option "No". If the user selects "Yes", the user can continue to answer the following questions, and if the user selects "No", the user ends this answer.

In the embodiment of the present application, if it is detected that the current active screening scene is present, the interactive interface 70 shown in FIG. 7A may be displayed. If it is detected that the current passive screening scene is present, the interactive interface 70 shown in FIG. 7B can be displayed. By asking whether it is the user who is coughing, it is avoided to recognize the coughing of the user's surroundings or other people. For this reason, it is necessary to consider whether the user is real or not. The case of coughing.

In one possible implementation manner, if an interactive interface is displayed through a display device, the user may be attracted to the display device by means of vibration, bright screen, prompt sound, and the like. If in the automatic screening scene, the user has already focused on the display device, then interact directly through the interactive interface, such as allowing the user to check the corresponding question. At this time, no voice output is required, and this application does not make specific limitations on this .

In the embodiment of the present application, the interactive element further includes a picture of sputum and an interactive control associated with the picture of sputum, and the interactive operation includes a touch operation acting on the interactive control.

Among them, the sputum pictures can be sputum pictures of different expectoration conditions, and different expectoration conditions can be no sputum, transparent sputum, rust-colored sputum, brick red sputum, golden yellow purulent sputum or The sputum is yellow-green purulent sputum, etc. For example, please refer to FIG. 7C together. The interactive interface 70 in FIG. 7C presents problems and corresponding pictures. The picture corresponding to option A in FIG. 7C is the picture without sputum. The picture corresponding to option B in Figure 7C is a picture of sputum with transparent sputum, the picture corresponding to option C in Figure 7C is a picture of sputum with rust-colored sputum, and the picture corresponding to option D in Figure 7C is a picture of sputum that is brick red The picture corresponding to option E in Figure 7C is a picture of golden yellow purulent sputum, and the picture corresponding to option F in Figure 7C is a picture of yellow-green purulent sputum.

Wherein, the interactive control associated with the sputum picture may be an option button under the sputum picture, such as options "A", "B", "C", "D", and "E" in FIG. 7C . When the user clicks an option, the sputum picture corresponding to the option is selected. The interactive control can also be a border set around the sputum picture, and the user clicks on the sputum picture or the border to select the sputum picture.

In the embodiment of the present application, the expectoration information represented by each sputum picture is different. If the picture corresponding to option A in FIG. 7C is selected, it will be recorded that this cough has no sputum. If the picture corresponding to option B in Figure 7C is selected, it is recorded that the sputum is transparent. If the picture corresponding to option C in Figure 7C is selected, it is recorded that the sputum is rust-colored. If the picture corresponding to option D in Figure 7C is selected, record that the sputum is brick red this time. If the picture corresponding to option E in Figure 7C is selected, it is recorded that the sputum is golden yellow purulent sputum. If the picture corresponding to option F in Figure 7C is selected, it is recorded that the sputum is yellow-green purulent sputum.

The difference between FIG. 7C and FIG. 7A is that each option in FIG. 7A is displayed in the form of a picture, which is convenient for the user to quickly determine.

In one possible implementation manner, the interactive element includes a viewfinder frame of the picture and a shooting control, and the interactive operation includes a touch operation acting on the shooting control.

For example, please refer to Fig. 8A, the interactive interface 70 presents a pop-up window, presents a question in the pop-up window, and asks the user: "Pictures of coughing up sputum are helpful for accurate screening of respiratory infections, helping you to further understand the risk situation and take corresponding measures, Please confirm whether you are allowed to take pictures to obtain your sputum information." The interactive interface 70 also includes three option buttons: A: no sputum, no need to take pictures; B: talk about it next time; C: agree to take pictures and upload information. If the user selects option A or B, you can prompt to thank the user for their feedback. If option C is selected, then enter the shooting interface 80, please also refer to FIG. Align the square frame with the sputum, or voice prompt the user to align the square frame with the sputum. The user clicks the close control 82 to exit the shooting interface. The user clicks on the shooting control 83 to take pictures. After taking pictures, the pictures are saved and uploaded to the cloud server or locally. The user can also use the built-in camera APP of the electronic device to take pictures, and then the user clicks the gallery control 84 to enter the gallery and select the corresponding sputum picture from the gallery. Please refer to FIG. 8C , the interactive interface 70 prompts the user "image acquisition is complete". A pop-up window can also appear on the interactive interface 70 to ask the user for other cough information, such as whether the sputum is viscous and whether the amount of sputum is large. For example, refer to FIG. Option A: more, B: no more, no less, C: no more, to get more information about coughing up sputum.

In this embodiment of the application, expectoration information can be obtained based on the sputum picture taken by the user combined with an image recognition algorithm, for example, the shape and color of the sputum can be analyzed to obtain the expectoration information. Exemplarily, for example, a sputum recognition model can be established to collect pictures of no sputum, transparent sputum, rust-colored sputum, brick red sputum, golden yellow purulent sputum, and Pictures of yellow-green purulent sputum, pictures with more sputum, pictures with less sputum, pictures with viscous sputum, and pictures with non-viscous sputum, and then train the learning model according to the collected pictures. The learning model after training is Sputum recognition model. The sputum picture uploaded by the user is input into the sputum recognition model for recognition, and finally the corresponding cough information can be obtained.

It can be understood that after obtaining the image information, the color and viscosity of the sputum can be recognized through the image, and the recognition model can be constructed by image segmentation and comparison with real expectoration pictures to obtain corresponding information. The image recognition process can be carried out locally on the device , can also be performed on a cloud server, which is not specifically limited in this application.

In the embodiment of the present application, in the active screening scenario, the user has already focused on the display device of the smart watch or mobile phone, and the user is directly prompted through the interactive interface to authorize the photo to be taken and uploaded. If it is a passive screening scenario, the presence of a wet cough (cough with phlegm) has been identified with a higher probability or a cough sound has been detected, but it may not necessarily be the user’s own cough, but may also come from the user’s surroundings or other people. It is necessary to take into account whether the user is really coughing, and use vibration, bright screen, prompt sound and other means to attract users to pay attention to the display device for interaction, such as voice or text pop-up asking "Did you cough just now?".

Step S404: Obtain expectoration information corresponding to the interactive operation.

In the embodiment of the present application, the sputum cough information includes: sputum cough frequency, sputum color, sputum viscosity, sputum volume, and the like. Among them, the frequency of coughing up sputum can be the number of times of coughing up sputum within a day, or the number of times of expectoration within a preset time, which can be determined by monitoring whether the coughing sound is detected, and whether the cough is dry or wet, to determine whether the coughing frequency is within the preset time. The frequency of coughing up sputum. Among them, the color of sputum can be transparent, rust-colored, brick-red, golden-yellow or yellow-green, etc. The consistency of sputum can be viscous or not viscous. Wherein, the amount of sputum may be the amount of sputum within a preset time, such as the amount of sputum in each expectoration, or the amount of sputum in a day.

Step S405: Outputting reminder information according to the user's physiological indicators and the sputum cough information.

In the embodiment of the present application, the expectoration information can be input into the respiratory infection screening algorithm, and then processed by the respiratory infection screening algorithm to determine whether the user's respiratory tract is abnormal or whether the user has a respiratory infection.

In this embodiment of the application, other inputs can be fused to assist in the screening of respiratory infection risks. Other inputs may include but are not limited to: physiological indicators (body temperature, breathing rate, heart rate, blood oxygen, etc.), user medical history, audio data (cough sounds, deep breath sounds, normal breath sounds, lung sounds, forced breath sounds, etc.), among them, the cough sounds in other inputs are used to screen respiratory infections, and the breath can be screened according to the pitch, loudness, frequency and other information of cough sounds Infect. Exemplarily, the electronic device detects the user's cough and records the user's cough sound, then starts to execute the screening method of the present application, obtains the user's sputum information, and obtains other inputs, such as the user's physiological indicators, the user's medical history, And the cough sound, deep breathing sound, normal breathing sound, lung sound, forced breathing sound, etc. input by the user again, and the obtained information is input into the respiratory infection screening method for processing.

In one possible implementation, the user's physiological indicators and sputum information can be input into the respiratory infection screening algorithm. The purpose of the respiratory infection screening algorithm is to help users judge What kind of respiratory infection risk situation is currently in, whether it is a mild infection such as the common cold or tonsillitis or a serious infection such as pneumonia? Guide users to self-intervene. Some studies even show that early warning can be given before the user's disease is initiated, so that users can pay attention It is very meaningful for special populations, a typical example is: the elderly with chronic obstructive pulmonary disease (COPD), after suffering from upper respiratory tract infection, if the condition is not controlled in time, COPD can occur Aggravation, followed by severe respiratory complications, life-threatening, if attention is paid at an earlier stage (the attention of the elderly themselves or their family members), it is hoped that the final risk can be controlled and the mortality rate can be reduced. On the other hand, in terms of epidemic prevention and control, if the degree of infection is found to be relatively mild, there is no need to go to the hospital to avoid contracting other epidemics.

The respiratory infection screening algorithm collects big data through the normal performance and abnormal performance of various information, covering the grouping of normal people and groups of people with various diseases, and constructs judgment rules to determine a fixed machine learning model. For example, the risk of a user's common cold can be judged by increasing body temperature, breathing rate, and cough frequency. The typical symptoms of various diseases can be used. The common cold usually does not cause fever, so the risk of common infection is low. After a design like this and big data modeling, a model (i.e. an algorithm) can be determined to assess a user's respiratory infection risk based on sensor data or user input.

Step S406: Outputting reminder information according to the user's physiological indicators and the coughing sound.

In the embodiment of the present application, if there is no sputum information, the user's physiological indicators and coughing sounds can be input into the respiratory infection screening algorithm to determine whether the user's respiratory tract is abnormal. The cough sound recorded in step S401 and the user's physiological index can be re-input into the respiratory infection screening algorithm, or the user's physiological index and the re-recorded cough sound can be input into the respiratory infection screening algorithm, or the additional auxiliary information (physiological Indicators, such as body temperature, respiration rate, heart rate, blood oxygen, etc., user medical history, audio data, such as cough sound, deep breath sound, normal breath sound, lung sound, forced breath sound, etc.) into the respiratory infection screening algorithm to judge the user's respiratory tract Are there exceptions.

In the embodiment of the present application, the function of expectoration information is to indicate the risk of certain serious diseases. For example, in pneumonia, bacterial infection is often accompanied by expectoration. Rust-colored sputum often indicates Streptococcus pneumoniae infection, brick-red sputum often indicates Klebsiella pneumoniae infection, golden-yellow purulent sputum often indicates Staphylococcus aureus infection, and yellow-green purulent sputum often indicates Pseudomonas aeruginosa infection. Infections with atypical pathogens such as Mycoplasma pneumoniae, Chlamydia pneumoniae, and Legionella pneumophila often manifest as dry cough and less sputum. Therefore, the appearance of expectoration with specific color and viscosity has the meaning of indicating risk. It helps to improve the final screening accuracy of the algorithm.

In the embodiment of the present application, the reminder information may include but not limited to: infection or not, risk level, risk of a specific disease, situation interpretation, advice, historical cough sound recording information, current or recent physiological index data, etc. Infection or not, that is, respiratory tract infection or respiratory tract non-infection, the risk level includes no abnormality or abnormality, that is, respiratory tract abnormality or non-abnormal, or risk level includes no abnormality, low, medium and high risk, that is, respiratory tract is abnormal abnormal, low risk of respiratory abnormality, medium risk of respiratory abnormality, and high risk of respiratory abnormality; risk of a particular disease, such as likelihood or risk of being influenza, likelihood or risk of being common cold, likelihood or risk of being pharyngitis, or Risk, the possibility or risk of laryngitis, the possibility or risk of pneumonia and the possibility or risk of bronchitis, etc., or the interpretation of the situation, the interpretation and suggestion of the user's recent data, the history of coughing sound recording information, the current or recent period Physiological index data of time, such as average body temperature and respiration rate, etc.

Please also refer to Figure 9A, the risk level displayed on the respiratory infection screening interface 50 is abnormal, and the situation interpretation and suggestion are: "By analyzing your recent measurements, you have a high risk of respiratory tract infection, please seek medical attention in time..." . The recorded cough sound has three segments of audio, and the collected physiological indicators include an average temperature of 37.1°C and a respiratory rate of 21 breaths per minute. Please also refer to Figure 9B. The risk level shown on the respiratory infection screening interface 50 is no abnormality. The situation interpretation and suggestion are: "By analyzing your recent measurements, your risk of respiratory tract infection is low. Please continue to be well." living habits...", the recorded cough sound has three segments of audio, the collected physiological indicators, the average temperature is 37.1 ℃, and the respiratory rate is 21 breaths per minute.

When the APP recognizes that the user's condition is more serious multiple times in a row, it can prompt the user whether he is willing to accept a return visit when the user opens the APP, so as to obtain expert help and answer questions. If the user authorizes and agrees, the sputum picture and the corresponding sputum color and viscosity information can be uploaded to the cloud, and the doctor decides on the intervention method based on the user's overall situation in the cloud and provides help to the user, including but not limited to telephone return visits and text chat interactions , SMS message, video intercom, etc. Physician return visits are not necessary, but with the deepening of digital medical care, the technology of smart terminal + sensor + doctor remote diagnosis mode continues to mature, and sputum information can be used in it.

In the embodiment of this application, it is mainly used to cooperate with the existing respiratory infection screening scenario, use the cough sound as the driving event, obtain the user's sputum information in time, and combine the sputum information with other inputs to perform respiratory infection screening , which can ensure the validity of expectoration information, the timeliness of screening and improve the overall screening accuracy.

The following introduces the active screening scenario of the embodiment of this application from the user's perspective:

The user wears a smart watch (or bracelet), which at least includes a microphone, a physiological data acquisition sensor, a speaker, a processor, a display screen, and a camera. The user installs a respiratory infection screening APP on the smart watch, which can be used for personal respiratory infection screening. The user clicks on the respiratory infection screening function in the respiratory infection screening APP, hoping to know his own respiratory infection risk in a short time. The respiratory infection screening APP prompts the user to sit still and wear the watch correctly. The smart watch acquires sensor data through physiological data The user's physiological data to obtain physiological indicators. The respiratory infection screening app prompts users to take a deep breath and then cough at least once. The user operates according to the prompt. The microphone of the watch records the cough sound, and the processor will process the cough sound. If it is not a cough, it will end and prompt the user to re-enter. Otherwise, it will identify whether it is a dry cough or a wet cough. If it is a dry cough, there will be no sputum information at this time. Cough sound and no sputum information are input as respiratory infection screening algorithm, and end; otherwise, it is a wet cough, and the user has sputum at this time, interact with the user through the display screen of the watch, or interact with the microphone and speaker to obtain the user's sputum Information, mainly focus on information such as sputum frequency, color, viscosity, etc., and input this information as a respiratory infection screening algorithm. Optional interaction methods include but are not limited to: The first method: the voice asks the user "Do you have sputum?" ", "Is the sputum sticky?", "What color is the sputum?" and other questions, according to the user's answer, the audio of the user's answer is converted into answer information to obtain the sputum information. The second method: the display screen of the watch presents typical sputum pictures or sputum color pictures with high screening effectiveness, allowing the user to choose the one that best suits the current situation, and no sputum is also an option. The third method: Prompt the user to take a photo of sputum, and obtain the cough information based on the photo of sputum combined with an image recognition algorithm. The sputum information is used as the input of the respiratory infection screening algorithm, and other inputs can be optionally fused to assist in screening the risk of respiratory infection and prompt information. Other inputs may include but are not limited to: physiological characteristics (body temperature, breathing rate, heart rate, blood oxygen, etc.) , user medical history, audio data (cough sounds, deep breath sounds, normal breath sounds, lung sounds, forced breath sounds, etc.).

In the active screening scenario, it is highly operable. The acquisition of sputum information is embedded in the entire screening process, which basically does not increase the time cost. The user's sputum information is obtained in the first time, and it is timely, accurate and directly used for breathing. Infection screening can improve screening accuracy.

The following introduces the passive screening scenario of the embodiment of this application from the user's perspective:

The user wears a smart watch (or bracelet), which at least includes a microphone, a physiological data acquisition sensor, a speaker, a processor, a display screen, and a camera. The user installs a respiratory infection screening APP on the smart watch, which can be used for personal respiratory infection screening. Check, the user authorizes the respiratory infection screening AP, so that the respiratory infection screening APP can run in the background and record the sound of the external environment in real time to obtain coughing sounds. The user clicks on the respiratory infection screening function in the respiratory infection screening APP, hoping to know his own respiratory infection risk in a short time. The smart watch monitors whether the user coughs through the microphone. If the user coughs, the microphone of the smart watch records the coughing sound and sends The cough sound is processed to identify whether it is a dry cough or a wet cough. If it is a dry cough, there is no sputum information at this time, and the cough sound and no sputum information are input as a screening algorithm to end; otherwise, it is a wet cough, and the smart watch passes Vibration, bright screen, prompt sound, etc. attract the user's attention and pay attention to the display screen. A pop-up window asks the user "Did you cough just now?" and asks the user whether he coughed. The main purpose is to avoid recording when other people around him cough. If there is no cough, it will end. Otherwise, the user has sputum at this time, and the user's sputum information is obtained through the smart watch, mainly focusing on information such as the frequency, color, and viscosity of the sputum, and this information is input as a respiratory infection screening algorithm, optional The interaction methods include but are not limited to: The first method: the voice asks the user "Is there phlegm?", "Is the phlegm sticky?", "What color is the phlegm?" Answer information to get sputum information. The second method: the display screen of the watch presents typical sputum pictures or sputum color pictures with high screening effectiveness, allowing the user to choose the one that best suits the current situation, and no sputum is also an option. The third method: Prompt the user to take a photo of sputum, and obtain the cough information based on the photo of sputum combined with an image recognition algorithm. The sputum information is used as the input of the respiratory infection screening algorithm, and other inputs can be optionally fused to assist in screening the risk of respiratory infection and prompt information. Other inputs may include but are not limited to: physiological characteristics (body temperature, breathing rate, heart rate, blood oxygen, etc.) , user medical history, audio data (cough sounds, deep breath sounds, normal breath sounds, lung sounds, forced breath sounds, etc.).

In the passive screening scenario, it is highly operable. The acquisition of sputum information is embedded in the entire screening process, which basically does not increase the time cost. The user's sputum information is obtained in the first time, and it is timely, accurate and directly used for breathing. Infection screening can improve the screening accuracy, and compared with the active screening scenario, the user does not feel it. The device performs background monitoring and prompts the user to provide information. The scene is stronger, and the background monitoring method further improves the timeliness of screening .

In the embodiment of this application, after acquiring the user's coughing sound, it can automatically judge whether it is a dry cough or a wet cough. It does not require the user to make a judgment on his own. It can be used in continuous monitoring scenarios, which maximizes the timeliness of screening. For wet cough, interact with users in a timely manner, record the user's expectoration information in a timely manner through various means, obtain expectoration information naturally, sequentially, and in line with the scene, and let users cooperate to obtain the first-time information in a convenient way. Hand-cough sputum information, reduce the interference of sputum-sputum information being forgotten or placed and deteriorated, and accurately obtain the latest information. The expectoration information is combined with other inputs in a timely manner to screen users for respiratory infections. After adding the expectoration information, the accuracy of screening can be improved, and the addition of this method basically does not affect the overall screening time and cost, and the entire process is natural. The screening results help users to intervene in the disease as soon as possible, for early detection and early intervention. Screen alarms in a timely manner that adapts to the scenario.

The descriptions of the processes corresponding to the above-mentioned figures have their own emphasis. For the parts not described in detail in a certain process, you can refer to the relevant descriptions of other processes.

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. When implemented using software, it may be implemented in whole or in part in the form of a computer program product. The computer program product for screening includes one or more computer instructions for screening. When these computer program instructions are loaded and executed on a computer, all or part of the process or function described in FIG. 4 according to the embodiment of the present application will be generated.

The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website, computer, server or data center Transmission to another website site, computer, server or data center by wired (eg coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (eg infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer, or a data storage device such as a server or a data center integrated with one or more available media. The available medium may be a magnetic medium (for example: floppy disk, hard disk, magnetic tape), an optical medium (for example: digital versatile disc (digital versatile disc, DVD)), or a semiconductor medium (for example: solid state disk (solid state disk, SSD) )Wait.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, and the like.

The above-mentioned embodiments provided by the application are not intended to limit the application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the application shall be included in the protection scope of the application. Inside.

Claims

A screening method, characterized in that the method comprises:

Detect the coughing sound, and perceive the user's interactive operation through the interactive device;

Obtaining expectoration information corresponding to the interactive operation;

Outputting reminder information according to the expectoration information.
The method according to claim 1, wherein the interaction device includes a voice interaction device, and the user's interaction operation perceived by the interaction device includes:

Outputting first voice data through the voice interaction device, wherein the first voice data is used to interact with the user;

receiving an interaction operation input by a user after the voice interaction device outputs the first voice data, wherein the interaction operation includes voice input.
The method according to claim 2, wherein, the first voice data includes guidance words set according to sputum information, and the guidance words include: general reply-type guidance words, check-type guidance Speech, Selective Guided Speech, and Inquiry Guided Speech.
The method according to claim 2 or 3, wherein the interaction device may further include a display device;

The display device displays text data and interactive elements converted from the first voice data;

Then the interactive operation further includes a touch operation acting on the interactive element.
The method according to claim 1, wherein the interactive device includes a display device, and the user's interactive operation perceived by the interactive device includes:

presenting an interactive interface through the display device, the interactive interface including interactive elements, wherein the interactive elements are used to interact with a user;

An interactive operation input by a user on the interactive interface is received.
The method according to claim 5, wherein the interactive element includes a first option set according to the expectoration information; then the interactive operation includes a touch operation acting on the first option.
The method according to claim 5 or 6, wherein the interactive elements further include sputum pictures and interactive controls associated with the sputum pictures;

Then the interactive operation includes a touch operation acting on the interactive control.
The method according to any one of claims 5 to 7, wherein the interactive elements include a viewfinder frame of a picture and shooting controls;

Then the interactive operation includes a touch operation acting on the shooting control.
The method according to any one of claims 1 to 7, wherein the sputum cough information includes: sputum cough frequency, sputum color and sputum viscosity.
The method according to any one of claims 1 to 9, wherein the outputting reminder information according to the expectoration information comprises:

Obtain the user's physiological indicators;

Outputting reminder information according to the physiological index and the expectoration information.
An electronic device, characterized in that it comprises:

memory for storing computer programs;

A processor, configured to execute the computer program stored in the memory, when the computer program is executed, the processor is configured to execute the method according to any one of claims 1 to 10.
A computer-readable storage medium, characterized in that the computer-readable storage medium includes computer instructions, and when the computer instructions are run on an electronic device, the electronic device performs any one of claims 1 to 10. the method described.
A computer program product, characterized in that the computer program product comprises computer program code, and when the computer program code is executed by a computer, the computer program code can cause the computer to perform any of claims 1 to 10. one method.