CN116013029A

CN116013029A - Fall detection method and device

Info

Publication number: CN116013029A
Application number: CN202111234148.8A
Authority: CN
Inventors: 于学猛
Original assignee: Hitachi Ltd
Current assignee: Hitachi Ltd
Priority date: 2021-10-22
Filing date: 2021-10-22
Publication date: 2023-04-25

Abstract

The invention provides a method and a device for detecting falling, and belongs to the technical field of falling detection. The falling detection method is applied to a radar module, the radar module comprises a plurality of radars, the radars comprise side-mounted radars and top-mounted radars, the side-mounted radars are mounted on the wall of a building, and the top-mounted radars are mounted on the top of the building, and the method comprises the following steps: detecting a target object in a building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring the data and the position information of the target object; sending the detection data to a server; receiving a first instruction sent by a server, and selecting a voice file corresponding to the first instruction from a plurality of prestored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server. The invention can accurately detect whether the target object falls down.

Description

Fall detection method and device

Technical Field

The invention relates to the technical field of fall detection, in particular to a fall detection method and device.

Background

With the increasing aging of the population, the safety of the elderly has become a problem that must be emphasized. Among them, falling is an important factor that endangers the health of the elderly. It is counted that more than 20% of men and 45% of women fall over among residents over 65 years old in China, and the older the age, the more likely the fall will occur. Because of uncertainty and unpredictability of falling, when the old falls, if the old cannot be effectively cured in time for a long time, long-term paralysis and even life threatening can be caused. Therefore, in order to ensure that the old people can be timely treated after falling, it is very important to perform falling detection on the old people.

Currently, a detection result of fall detection still has a large error.

Disclosure of Invention

The invention aims to solve the technical problem of providing a method and a device for detecting falling, which can accurately detect whether a target object falls.

In order to solve the technical problems, the embodiment of the invention provides the following technical scheme:

in one aspect, a fall detection method is provided, applied to a radar module, the radar module including a plurality of radars, the plurality of radars including a side-mounted radar and a top-mounted radar, the side-mounted radar being mounted on a wall of a building, the top-mounted radar being mounted on a top of the building, the method comprising:

detecting a target object in the building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring data and the position information of the target object;

transmitting the detection data to a server;

receiving a first instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play;

and receiving the voice signal returned by the target object and sending the voice signal to the server.

In an optional embodiment of the present invention, the receiving the voice signal returned by the target object and sending the voice signal to the server includes:

monitoring a voice signal of the target object within the duration indicated by the first instruction;

and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

In an optional embodiment of the present invention, after the step of receiving the voice signal returned by the target object and sending the voice signal to the server, if the content of the voice data is empty, the method further includes:

receiving a repeated playing instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of prestored voice files to play;

In an alternative embodiment of the invention, the method further comprises at least one of:

receiving a deleting instruction of the server, and deleting the stored voice file corresponding to the deleting instruction;

Receiving an updating instruction of the server, and updating a stored voice file corresponding to the updating instruction;

and receiving the voice file sent by the server and storing the voice file.

The embodiment of the invention also provides a fall detection method which is applied to the server and comprises the following steps:

receiving detection data sent by a radar module, wherein the radar module comprises a plurality of radars, and the detection data comprises the state of a target object, the number of the radars for acquiring the data and the position information of the target object;

determining a radar corresponding to first detection data, wherein the state of a target object in the first detection data is falling;

a first instruction is sent to the radar, and the radar is instructed to select a voice instruction corresponding to the first instruction from a plurality of prestored voice files to play;

and receiving a voice signal sent by the radar, and determining whether the target object falls down according to the voice signal.

In an alternative embodiment of the present invention, determining whether the target object falls based on the speech signal comprises:

decoding the voice signal to obtain information comprising a radar number, an identification of the first instruction, a number of voice data and content of the voice data;

If the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play;

and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

In an alternative embodiment of the present invention, the method further comprises:

receiving a personalized voice setting instruction sent by a mobile terminal;

and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

transmitting a deleting instruction to the radar module, and indicating to delete the voice file corresponding to the deleting instruction;

and sending an update instruction to the radar module to indicate to update the voice file corresponding to the update instruction.

The embodiment of the invention also provides a falling detection device, which is applied to a radar module, wherein the radar module comprises a plurality of radars, the radars comprise side-mounted radars and top-mounted radars, the side-mounted radars are arranged on the wall of a building, and the top-mounted radars are arranged on the top of the building, and the device comprises:

The detection module is used for detecting a target object in the building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring data and the position information of the target object;

the communication module is used for sending the detection data to a server; receiving a first instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

In an optional embodiment of the present invention, the communication module is specifically configured to monitor a voice signal of the target object within a duration indicated by the first instruction; and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

In an optional embodiment of the present invention, the communication module is further configured to receive a repeat play instruction sent by the server, and select, from a plurality of pre-stored voice files, a voice file corresponding to the first instruction for playing; and receiving the voice signal returned by the target object and sending the voice signal to the server.

In an alternative embodiment of the invention, the communication module is further configured to perform at least one of:

and receiving the voice file sent by the server and storing the voice file.

The embodiment of the invention also provides a falling detection device which is applied to the server and comprises:

the communication module is used for receiving detection data sent by the radar module, the radar module comprises a plurality of radars, and the detection data comprises the state of a target object, the number of the radars for acquiring the data and the position information of the target object;

the processing module is used for determining a radar corresponding to first detection data, wherein the state of a target object in the first detection data is falling;

the communication module is also used for sending a first instruction to the radar and instructing the radar to select a voice instruction corresponding to the first instruction from a plurality of prestored voice files to play; receiving a voice signal sent by the radar;

the processing module is also used for determining whether the target object falls down according to the voice signal.

In an optional embodiment of the present invention, the processing module is specifically configured to decode the voice signal to obtain information including a radar number, an identifier of the first instruction, a number of voice data, and a content of the voice data; if the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

In an optional embodiment of the present invention, the communication module is further configured to receive a personalized voice setting instruction sent by the mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

The embodiment of the invention also provides fall detection equipment, which comprises:

A processor; and

a memory in which computer program instructions are stored,

wherein the computer program instructions, when executed by the processor, cause the processor to perform the steps in the fall detection method as described above.

Embodiments of the present invention also provide a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps in the fall detection method as described above.

The embodiment of the invention has the following beneficial effects:

in the scheme, after the radar detects that the target object falls, the radar can be used for playing a voice file to the target object, and whether the target object falls is confirmed through voice interaction, so that whether the target object falls can be accurately detected.

Drawings

Fig. 1 is a flow chart of a fall detection method applied to a radar module according to an embodiment of the invention;

FIG. 2 is a schematic view of a side-mounted radar according to an embodiment of the present invention;

FIG. 3 is a schematic diagram of a top-loading radar according to an embodiment of the present invention;

fig. 4 is a flowchart of a fall detection method applied to a server according to an embodiment of the present invention;

FIG. 5 is a schematic diagram of a system architecture according to an embodiment of the present invention;

fig. 6 is a schematic flow interaction diagram of a fall detection method according to an embodiment of the present invention;

fig. 7 is a block diagram of a fall detection device applied to a radar module according to an embodiment of the present invention;

fig. 8 is a block diagram of a fall detection device applied to a server according to an embodiment of the present invention;

fig. 9 is a block diagram of a fall detection apparatus according to an embodiment of the invention.

Detailed Description

In order to make the technical problems, technical solutions and advantages to be solved by the embodiments of the present invention more apparent, the following detailed description will be given with reference to the accompanying drawings and the specific embodiments.

The embodiment of the invention provides a method and a device for detecting falling, which can accurately detect whether a target object falls.

Example 1

The embodiment of the invention provides a fall detection method, which is applied to a radar module, wherein the radar module comprises a plurality of radars, the radars comprise side-mounted radars and top-mounted radars, the side-mounted radars are arranged on a wall of a building, and the top-mounted radars are arranged on the top of the building, as shown in fig. 1, and the method comprises the following steps:

step 101: detecting a target object in the building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring data and the position information of the target object;

In this embodiment, the building can be the house of nursing home or old man living, and the target object can be the old man that needs guardianship, is provided with a plurality of radars in the building, and the radar can adopt millimeter wave radar, and this embodiment uses millimeter wave radar long-range detection old man's action, does not collect organism sign data such as face, and influence such as illuminance can 24 hours use round clock, neither can bring inconvenience for the old man like wearing equipment, also can not bring the psychological burden that privacy was revealed for the old man like the camera.

In this embodiment, the radar includes a side-mounted radar and a top-mounted radar, both of which are located inside a building, and can detect a target object. As shown in fig. 2, the side-mounted radar LD1 is installed on a wall of a building; as shown in fig. 3, the roof-mounted radar LD2 is installed at the top of a building; according to the embodiment, the side-mounted radar and the top-mounted radar are combined, so that the side-mounted radar and the top-mounted radar can be flexibly arranged according to specific scenes, and diversified customer scenes and customized installation can be met to the maximum extent. The side-mounted radars can be arranged on each wall of the building, and are used for collecting action tracks and position data of the old and judging falling; the roof radar can be multiple, is installed in different areas at the top of a building, and is used for collecting action tracks and position data of old people and judging falling. In order to ensure that the detection range can cover the whole building, the detection areas of the top-mounted radar and the side-mounted radar should cover the whole building, and the detection areas between adjacent radars have overlap.

In this embodiment, N millimeter wave radars may be installed in each room, N being an integer greater than 1, and after each millimeter wave radar is installed, the installation position thereof with respect to the entire building and the relative coordinate information with respect to the important object such as the wall surface, door, etc. of the room where it is located may be measured. After the installation of the N millimeter wave radars is completed, as shown in fig. 5, the installation position of each radar and the position information near the radar need to be sent to a cloud server, the cloud server can communicate with side-mounted radars (i.e., side-mounted radars) and top-mounted radars (top-mounted radars), and a track fusion module, a voice recognition and dialogue module, a fall comprehensive judgment module, a personalized voice processing module and an alarm module are arranged at the cloud server. The alarm module is connected with the mobile terminal and the monitoring center and can send alarm information to the mobile terminal and the monitoring center.

When the system of the embodiment works, all millimeter wave radars in a room are started, when an old man enters the room, the millimeter wave radars mmwave_n in the room can capture the relative position of a human body relative to the millimeter wave radars, whether the old man falls down or not is judged in 2s, information is sent to a data center of a cloud server end, the data format is [ mmwave_n (namely a radar number), status (namely the state of a target object), ref_x, ref_y, ref_z ], wherein [ ref_x, ref_y, ref_z ] is the three-dimensional coordinate of the target object in a radar coordinate system, and the position information of the target object is represented. The millimeter wave radar in the room can continuously output the data to the data center as long as the old people are in the room, and if the old people leave the room, the millimeter wave radar cannot detect the old people, and the data transmission to the data center can be stopped. If the old people leave the current room and enter another room, the millimeter wave radar in the other room continues to detect, and the detection data is sent to the data center.

In this embodiment, the positional relationship between the target object and the radar may be: facing the radar, facing away from the radar, being located on the right side of the radar and being located on the left side of the radar.

In this embodiment, a fall refers to a sudden, involuntary, unintended posture change, falling to the ground or onto a lower plane. Reasons for a fall include: ground slippery, stumbling with obstacles, light deficiency, gait disorders (etiology, toddler gait, etc.).

The radar can determine whether the target object falls by detecting the following body parts: knee, calf, thigh, hip, back, abdomen, chest, shoulder, elbow, head, hand.

When a fall occurs, the body part of the target subject may be one of the following:

double kneeling falls, such as a sudden stumbled forward fall during walking, with both hands holding the sofa right in front (quick fall); or, the heart attack kneels down slowly, but does not fall to the ground completely (slow fall) due to the cabinet in front; or, the heart attack kneels slowly, with both hands continuously placed at the chest (slow fall);

a double kneeling land falls with both hands on the ground, such as when the hands are on the ground after walking over an open area and stumbled over the kneeling land (rapid fall); or, after a heart attack, kneeling slowly, the hands are supported on the ground (slow fall);

The user falls over by double kneeling, and the left hand is propped against the ground, for example, the user walks on the tea table side to trip suddenly, the left hand is propped against the ground, and the right hand is propped against the tea table side (fast falling); or, after the heart attack slowly kneels down, the left hand is supported on the ground, and the right hand holds the chest (slowly falls down);

the two-knee kneeling falls, the right single hand is propped against the ground, such as when the user walks on the tea table and suddenly stumbles, the right hand is propped against the ground, the left hand is propped against the tea table (fast falling), or the right hand is propped against the ground after the heart disease attacks slowly kneeling, and the left hand is propped against the chest (slow falling);

the head falls down on the ground when the two knees kneel, such as a small stool is stepped on to fall forward suddenly to knock the head (fast fall), or the head falls down to the ground after the two hands kneel and cover the head to fall (slow fall);

the left knee falls down on the kneeling place, for example, between a sofa and a tea table, due to the narrow place, the body of the left knee leans against the sofa after falling down (fast falling), or the right leg is blocked when the crutch walks at a slow speed, and the crutch is held by both hands after being stumbled (slow falling);

the right knee falls down on the kneeling place, for example, between a sofa and a tea table, due to the narrow place, the body of the right knee leans against the sofa after falling down (fast falling), or the left leg is blocked when the crutch walks at a slow speed, and the crutch is held by both hands after being stumbled (slow falling);

The left knee falls on the kneeling place, and the two hands are on the ground, for example, the shoelaces stepped on the right foot when walking are stumbled and the two hands are on the ground (fast fall), or the left leg is powerless to support and kneel on the ground when the right leg is suddenly painful to look at (slow fall);

the right knee falls over on a kneeling place, and the two hands are on a ground, for example, the two hands are on a ground after the shoelaces stepped on the left foot are stumbled when walking (fast fall), or the right leg is weak to support and kneel on the ground when the right leg is suddenly painful to look at (slow fall);

the left knee falls down on the kneeling ground, and the left hand props the ground;

the left knee falls down on the kneeling ground, and the right hand falls on the ground;

the right knee falls over on a kneeling place, and the left hand falls on a supporting place;

the right knee falls over on a kneeling place, and the right hand props up the ground;

sitting falls, two legs land, such as when the ground falls wet and suddenly is slipped down to sit on tables on two sides of the ground (rapid fall), and the two legs fall between a living room, a television cabinet and a tea table; or, the hands holding the head and back of the wall or the furniture slowly slip down and sit on the ground (slowly fall) for the sudden brain diseases;

sitting and falling, the two hands of the two legs are grounded, for example, as the ground is wet, slippery and falling and has no thing which can be held, the two hands are supported on the ground (rapid falling); or, the sudden illness is backed up against the wall or the furniture is supported on the ground by both hands after slowly sliding down (falling down slowly);

Sitting and falling, the left hand with two legs and one hand is grounded, for example, the right hand falls backwards and falls to a desk beside due to the wet ground, and the left hand is supported on the ground (fast falling); or, the furniture is slowly slipped back against the heart disease, and the right hand is covered on the chest and the left hand is supported on the ground (slowly fallen down);

sitting and falling, the two legs are grounded by right and one hand, for example, the ground is wet and slippery, the left hand falls backwards and is supported by the table beside, and the right hand is supported on the ground (fast falling); or, the furniture is slowly slipped back against the heart disease, the left hand is covered on the chest and the right hand is supported on the ground (slowly fallen down);

falls on the ground, left Shan Tui, left one hand is grounded;

fall on the ground, the right single leg and the right single hand;

left side falls, such as left side fall due to sole slip caused by small articles stepping on the ground, left hand left leg side first land (rapid fall);

the right side falls, for example, the sole slips and falls to the right side due to the small article stepped on the ground, and the right leg of the right hand lands on the ground (falls quickly);

the user falls on the back, the legs are straightened, the head is straightened, for example, the ground of a bathroom is slippery, and the whole user is straight and falls backwards (falls quickly);

the user falls on the back, the legs straighten, the head lifts, for example, the floor of a bathroom is wet and slippery, the whole user falls backwards but stretches the hand to pull the handrail in front of the handrail, and the head falls on the ground (rapid falling); or, the bathroom floor is slippery, the whole person falls backwards but wants to reach the armrest before pulling out the hand in an attempt to prevent falling (rapid falling);

The legs are bent when the user falls on the back, for example, the user falls backwards after suddenly hitting the legs during the fast walking process, the legs are curled and lifted after lying, and the two hands hold the hit legs (fast falling);

the user falls on the back, the left single leg bends, for example, the user falls backwards after suddenly hitting the foot during the fast walking process, the left leg curls and lifts up after lying, and the user holds the hit left leg with both hands (fast falling);

the user falls on the back, the right single leg bends, for example, the user falls backwards after suddenly hitting the foot during the fast walking process, the right leg curls and lifts up after lying, and the user holds the hit right leg with both hands (fast falling);

front fall, leg straightening, head straightening, such as sudden syncope in open areas followed by a forward fall (rapid fall);

the front falls, the legs straighten, and the head is lifted up, for example, when the user walks rapidly, the user suddenly stumbles, and the head of the small stool which is held in front by the hand when the user falls forward is not landed (rapid falling); or, when the user walks quickly, the user suddenly stumbles, and consciously, when the user falls forward, the user props the head on the ground and does not land (quickly falls down);

the front falls, the legs are lifted, the head straightens, for example, when the user walks rapidly, the user suddenly stumbles over the case, and the legs are supported on the case after the user falls forwards (the user falls rapidly);

The front falls, the left single leg is lifted, the head straightens, for example, the head suddenly stumbles over the box when walking quickly, and the left leg is supported on the box after falling forwards (quick falling);

the front falls, the right single leg is lifted, the head straightens, for example, when walking fast, the head suddenly stumbles over the box, and the right leg is supported on the box after falling forward (fast falling).

In this embodiment, the radar detects the moving speed, moving direction, falling speed, motion, and holding of the target object, where the motion includes: walking, squatting, bending down, kneeling, the action that the target subject may transfer, sitting down, lying down, standing. The fall requires measurement of basic data: slow walking speed, normal walking speed, tripping time, general falling time, slow falling time by walking, etc.

Step 102: transmitting the detection data to a server;

the radar may transmit the detection data to the server through wireless communication.

Step 103: receiving a first instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play;

the server receives detection data of a plurality of radars, can determine first detection data from the detection data, the state of a target object of the first detection data is falling, and sends a first instruction to the corresponding radar according to the radar number in the first detection data so as to confirm whether the target object truly falls.

In this embodiment, the radar stores a plurality of voice files, and different voices can be played to the target object through each voice file, for example, ask the target object "please ask you to fall down" or the like. The first instruction can carry the identification of the voice file, and the corresponding voice file can be searched and played according to the identification of the voice file.

Step 104: and receiving the voice signal returned by the target object and sending the voice signal to the server.

In the embodiment, the radar selects a designated voice file from a plurality of question voice files built in the radar according to the instruction of the server, and simultaneously opens a microphone, monitors the sound in a room, and synchronously carries out noise reduction and echo cancellation processing on the received microphone voice; monitoring a voice signal of the target object within the duration indicated by the first instruction, closing a microphone after monitoring, encoding and compressing the voice signal, packaging and sending the voice signal to the server, wherein the information sent to the server comprises a radar number, an identification of the first instruction, a number of voice data and content of the voice data, and specifically, the formats are [ mmwave_n (radar number), command_id (identification of the first instruction), sound_n (number of the voice data) and sound_contents (content of the voice data) ].

If the content of the voice data is empty or does not meet the requirement, receiving a repeated playing instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server. This step is repeated until the content of the speech data meets the requirements or is not empty or the waiting time exceeds a threshold.

In addition, according to the requirement of a user, the radar can also receive a deleting instruction of the server and delete the stored voice file corresponding to the deleting instruction; or receiving an update instruction of the server, and updating a stored voice file corresponding to the update instruction; or receiving the voice file sent by the server and storing the voice file.

In this embodiment, the voice file stored in the radar not only includes the content of asking you to do a fall, but also includes the content of asking for an injury, reminding the target object to take medicine, paying attention to the injury, and the like, and the voice file stored in the radar not only can perform voice interaction with the target object, but also can remind the target object to perform corresponding operations according to the body data of the target object, such as taking medicine on time, taking a rest on time, and the like, so as to ensure the physical health of the target object.

The embodiment of the invention also provides a fall detection method, which is applied to the server, as shown in fig. 4, and comprises the following steps:

step 201: receiving detection data sent by a radar module, wherein the radar module comprises a plurality of radars, and the detection data comprises the state of a target object, the number of the radars for acquiring the data and the position information of the target object;

in this embodiment, it is necessary to determine whether the radar transmitting the detection data is a side-mounted radar or a top-mounted radar according to the radar number in the detection data, and if the radar transmitting the detection data is a side-mounted radar, convert the position information in the detection data according to the installation angle of the side-mounted radar.

As shown in fig. 5, the track fusion module receives all the detection data reported by the radars, identifies whether the received detection data is the side-mounted radar or the top-mounted radar according to the radar number, if the received detection data is the detection data of the side-mounted radar, converts the position information in the detection data according to the installation angle of the side-mounted radar, such as a 15-degree dip angle, and then clusters and tracks the coordinate data of the side-mounted radar or the top-mounted radar after the conversion. The coordinates uploaded to the server by the radar are all based on the coordinate system data with the current radar as an origin, and the coordinates are required to be transformed based on the position coordinates installed in the whole house, so that the coordinate data under the unified coordinates of the whole house are obtained; fusing and/or clustering the plurality of coordinate data, and removing the coincident and invalid coordinate data to obtain a plurality of valid coordinate data; and acquiring the action track of the target object according to the plurality of effective coordinate data. Specifically, a kalman filtering mode can be adopted to obtain the action track of the target object, and relay and smooth filtering of tracking marks among multiple radars are required to be realized.

Step 202: determining a radar corresponding to first detection data, wherein the state of a target object in the first detection data is falling;

Step 203: a first instruction is sent to the radar, and the radar is instructed to select a voice instruction corresponding to the first instruction from a plurality of prestored voice files to play;

in this embodiment, the radar stores a plurality of voice files, and different voices can be played to the target object through each voice file, for example, ask the target object "please ask you to fall down" or the like. The first instruction can carry the identification of the voice file, and the radar can be instructed to find and play the corresponding voice file according to the identification of the voice file.

Step 204: and receiving a voice signal sent by the radar, and determining whether the target object falls down according to the voice signal.

The information sent by the radar to the server includes a radar number, an identification of the first instruction, a number of the voice data, and a content of the voice data, specifically, a format of [ mmwave_n (radar number), command_id (identification of the first instruction), sound_n (number of the voice data), sound_contents (content of the voice data) ].

The server decodes the voice signal to obtain information comprising a radar number, an identification of the first instruction, a number of voice data and content of the voice data, wherein the format is [ mmwave_n (radar number), command_id (identification of the first instruction), sound_n (number of the voice data), sound_contents (content of the voice data) ]; and carrying out voice-to-text processing on the content of the replied voice data, calling a dialogue control logic module, understanding the replied content and deciding the next dialogue logic, wherein the dialogue control logic module comprises algorithms such as logic reasoning based on a knowledge base, deep learning based on RNN and the like, and the like to realize question-answer content matching. If the reply content is empty (i.e. the target object is not answered) or cannot be identified, starting repeated questioning logic according to a strategy, sending repeated playing instructions to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; if the content of the voice data is not empty, determining whether the target object falls according to the content of the voice data, generating a voice judgment result, and sending the voice judgment result to a fall comprehensive judgment module, wherein the voice judgment result is in the form of [ mmwave_n, status, ref_x, ref_y, ref_z ]. For example, if the content of the voice data is "i have not fallen down", it can be judged that the target object has not fallen down; if the content of the voice data is "me fallen", it can be judged that the target object does fall.

In this embodiment, the server may also receive a personalized voice setting instruction sent by the mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module. The server monitors and receives the data of the mobile terminal, if the user does not perform personalized voice setting, the mobile terminal can prompt the user whether the personalized voice setting is needed, if the user agrees, the personalized voice setting logic is entered, personalized contents comprise voice sound, dialogue contents, playing time and the like, the user can select from candidate modes (such as voice sound modes), and the user can autonomously set the modes (such as submitting a recording file) to perform personalized sound generation. After setting personalized voice sound, dialogue content, playing plan and the like, the voice-print voice dialogue content can be generated by utilizing three-party voice synthesis service, waveform splicing, parameter synthesis, an end-to-end voice synthesis algorithm and the like, and the generated personalized voice-print dialogue file, dialogue plan and the like are submitted to a voice recognition and dialogue module to update the personalized voice file at the radar end.

In this embodiment, the server may further send a deletion instruction to the radar module, to instruct to delete the voice file corresponding to the deletion instruction; or sending an update instruction to the radar module to indicate to update the voice file corresponding to the update instruction.

In this embodiment, the voice file sent to the radar by the server not only includes "ask you about the fact that you have fallen down" but also includes asking for injury, reminding the target object to take medicine, paying attention to injury, and the like.

After confirming that the target object falls, an alarm can be given to the monitoring center. The alarm priority can be determined according to the current time information, the falling mode and the body data of the target object, for example, the alarm priority is higher when the current time is at night, and the alarm priority is lower when the current time is at daytime; the falling mode is rapid falling, the alarm priority is higher, the falling mode is slow falling, and the alarm priority is lower; the body data of the guardian show that the physical condition is good, and the alarm priority is lower; the body data of the guardian show that the body condition is poor, and the alarm priority is higher; after falling, the guardian stands up quickly (stands up to sit on the sofa within 30 seconds), and the alarm priority is lower; the guardian stands up after falling for a long time (stands up after 1 minute, the stationary signal disappears in the process, or stands up after 1 minute, the leg knees are massaged in the process), and the alarm priority is higher.

According to the alarm priority, searching an alarm mode corresponding to the preset alarm priority to alarm, such as log record, general notification, emergency short message notification, emergency telephone voice notification, emergency manual real-time intervention and the like, and starting different alarm methods.

For the mobile terminal, the server-side data can be called according to the user operation, the page and the content can be generated, and the main functions of the mobile terminal are as follows: a user state confirmation function of confirming the state of the person under guardianship, such as a dangerous state of falling down; if the voice replying voiceprint file exists, calling a voice player to listen to voice content; the remote real-time conversation function can be started to carry out real-time conversation with the target object through the radar according to the requirement; and the user personalized management function comprises checking, generating and deleting dialogue records, dialogue contents, playing mechanisms and the like which are preset on the radar or the server and used for talking with the guardian. The mobile terminal can also respond to the user operation of the mobile terminal to call corresponding background functions, such as checking the state of the monitored old (including playing of voice reply sound files and the like), historical data and the like.

Fig. 6 is a flow interaction schematic diagram of a fall detection method according to an embodiment of the present invention, as shown in fig. 6, where the embodiment includes:

Step 601: the radar detects that the target object falls down and sends detection data to the server;

when the system of the embodiment works, all millimeter wave radars in a room are started, when an old man enters the room, the millimeter wave radars mmwave_n in the room can capture the relative position of a human body relative to the millimeter wave radars, judge whether the old man falls down in 2s, send information to a server, and the data format is [ mmwave_n (namely the radar number), status (namely the state of a target object), ref_x, ref_y, ref_z ], wherein [ ref_x, ref_y, ref_z ] is the three-dimensional coordinate of the target object under a radar coordinate system, and represents the position information of the target object. The millimeter wave radar in the room will continuously output the data to the server as long as the old people are in the room, and if the old people leave the room, the millimeter wave radar cannot detect the old people, and the data transmission to the server will be stopped. If the old people leave the current room and enter another room, the millimeter wave radar in the other room continues to detect, and the detection data is sent to the server.

Step 602: the server starts a voice function and sends a first instruction to the radar;

Step 603: the radar initiates a voice conversation while turning on the microphone;

the radar selects a specified voice file from a plurality of question voice files built in the radar to play according to the instruction of the server, and simultaneously opens a microphone to monitor the sound in the room.

Step 604: the radar receives the reply sound of the target object and sends a voice signal to the server;

the radar carries out noise reduction and echo cancellation on the received microphone voice; monitoring a voice signal of the target object within the duration indicated by the first instruction, closing a microphone after monitoring, encoding and compressing the voice signal, packaging and sending the voice signal to the server, wherein the information sent to the server comprises a radar number, an identification of the first instruction, a number of voice data and content of the voice data, and specifically, the formats are [ mmwave_n (radar number), command_id (identification of the first instruction), sound_n (number of the voice data) and sound_contents (content of the voice data) ].

Step 605: the server identifies the reply content, determines whether to continue the session, if not, goes to step 606, if yes, goes to step 602;

The server decodes the voice signal to obtain information comprising a radar number, an identification of the first instruction, a number of voice data and content of the voice data, wherein the format is [ mmwave_n (radar number), command_id (identification of the first instruction), sound_n (number of the voice data), sound_contents (content of the voice data) ]; and carrying out voice-to-text processing on the content of the replied voice data, calling a dialogue control logic module, understanding the replied content and deciding the next dialogue logic, wherein the dialogue control logic module comprises algorithms such as logic reasoning based on a knowledge base, deep learning based on RNN and the like, and the like to realize question-answer content matching. If the reply content is empty (i.e. the target object is not answered) or cannot be identified, starting repeated questioning logic according to a strategy, turning to step 602, retransmitting a first instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; if the content of the voice data is not empty, go to step 606.

Step 606: and the server alarms according to the reply content.

And determining whether the target object falls according to the content of the voice data, generating a voice judgment result, and sending the voice judgment result to a fall comprehensive judgment module, wherein the voice judgment result is in the form of [ mmwave_n, status, ref_x, ref_y, ref_z ]. For example, if the content of the voice data is "i have not fallen down", it can be judged that the target object has not fallen down; if the content of the voice data is "me fallen", it can be judged that the target object does fall. After confirming that the target object falls, an alarm can be given to the monitoring center. The alarm priority can be determined according to the current time information, the falling mode and the body data of the target object, and according to the alarm priority, the alarm mode corresponding to the preset alarm priority is searched for alarm, such as log record, general notification, emergency short message notification, emergency telephone voice notification, emergency manual real-time intervention and the like, and different alarm methods are started.

In this embodiment, after the radar detects that the target object falls, the radar is further used to play a voice file to the target object, and through voice interaction, it is confirmed whether the target object falls, so that whether the target object falls can be accurately detected.

Example two

The embodiment of the invention also provides a fall detection device, which is applied to a radar module, wherein the radar module comprises a plurality of radars, the radars comprise a side-mounted radar and a top-mounted radar, the side-mounted radar is arranged on a wall of a building, and the top-mounted radar is arranged on the top of the building, as shown in fig. 7, and the device comprises:

a detection module 31, configured to detect a target object in the building, and obtain detection data, where the detection data includes a state of the target object, a number of a radar for acquiring data, and position information of the target object;

a communication module 32 for transmitting the detection data to a server; receiving a first instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

In an alternative embodiment of the present invention, the communication module 32 is specifically configured to monitor a voice signal of the target object within a duration indicated by the first instruction; and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

In an alternative embodiment of the present invention, the communication module 32 is further configured to receive a repeat play command sent by the server, and select, from a plurality of pre-stored voice files, a voice file corresponding to the first command to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

In an alternative embodiment of the present invention, the communication module 32 is further configured to perform at least one of:

and receiving the voice file sent by the server and storing the voice file.

The embodiment of the invention also provides a fall detection device, which is applied to a server, as shown in fig. 8, and comprises:

a communication module 41, configured to receive detection data sent by a radar module, where the radar module includes a plurality of radars, and the detection data includes a state of a target object, a number of the radars that acquire the data, and position information of the target object;

a processing module 42, configured to determine a radar corresponding to first detection data, where a state of a target object in the first detection data is a fall;

the communication module 41 is further configured to send a first instruction to the radar, instruct the radar to select a voice instruction corresponding to the first instruction from a plurality of pre-stored voice files to play; receiving a voice signal sent by the radar;

the processing module 42 is further configured to determine whether the target object falls based on the speech signal.

In an alternative embodiment of the present invention, the processing module 42 is specifically configured to decode the voice signal to obtain information including a radar number, an identifier of the first instruction, a number of voice data, and a content of the voice data; if the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

In an alternative embodiment of the present invention, the communication module 41 is further configured to receive a personalized voice setting instruction sent by the mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

In an alternative embodiment of the present invention, the communication module 41 is further configured to perform at least one of:

In the embodiment of the application, after the radar detects that the target object falls, the radar is used for playing the voice file to the target object, and whether the target object falls is confirmed through voice interaction, so that whether the target object falls can be accurately detected.

Example III

The embodiment of the invention also provides a fall detection device 50, as shown in fig. 9, comprising:

a processor 52; and

a memory 54, in which memory 54 computer program instructions are stored,

in some embodiments, the fall detection device 50 is applied to a radar module comprising a plurality of radars including a side-mounted radar mounted on a wall of a building and a top-mounted radar mounted on top of the building, which when executed by the processor causes the processor 52 to perform the steps of:

transmitting the detection data to a server;

In some embodiments, the fall detection device 50 is applied to a server, which when executed by the processor, causes the processor 52 to perform the steps of:

Further, as shown in fig. 9, the fall detection device 50 further includes a network interface 51, an input device 53, a hard disk 55, and a display device 56.

The interfaces and devices described above may be interconnected by a bus architecture. The bus architecture may be a bus and bridge that may include any number of interconnects. One or more Central Processing Units (CPUs), represented in particular by processor 52, and various circuits of one or more memories, represented by memory 54, are connected together. The bus architecture may also connect various other circuits together, such as peripheral devices, voltage regulators, and power management circuits. It is understood that a bus architecture is used to enable connected communications between these components. The bus architecture includes, in addition to a data bus, a power bus, a control bus, and a status signal bus, all of which are well known in the art and therefore will not be described in detail herein.

The network interface 51 may be connected to a network (e.g., the internet, a local area network, etc.), and may obtain relevant data from the network and may be stored in the hard disk 55.

The input device 53 may receive various instructions from an operator and send them to the processor 52 for execution. The input device 53 may comprise a keyboard or a pointing device (e.g. a mouse, a trackball, a touch pad or a touch screen, etc.).

The display device 56 may display results from the execution of instructions by the processor 52.

The memory 54 is used for storing programs and data necessary for the operation of the operating system, and data such as intermediate results in the calculation process of the processor 52.

It will be appreciated that the memory 54 in embodiments of the invention may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The nonvolatile memory may be Read Only Memory (ROM), programmable Read Only Memory (PROM), erasable Programmable Read Only Memory (EPROM), electrically Erasable Programmable Read Only Memory (EEPROM), or flash memory, among others. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. The memory 54 of the apparatus and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

In some implementations, the memory 54 stores the following elements, executable modules or data structures, or a subset thereof, or an extended set thereof: an operating system 541 and application programs 542.

The operating system 541 includes various system programs, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks. The application programs 542 include various application programs such as a Browser (Browser) and the like for implementing various application services. A program for implementing the method of the embodiment of the present invention may be included in the application program 542.

In some embodiments, the processor 52, when calling and executing the application programs and data stored in the memory 54, specifically performs the following steps: detecting a target object in the building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring data and the position information of the target object; transmitting the detection data to a server; receiving a first instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

The method disclosed in the above embodiment of the present invention may be applied to the processor 52 or implemented by the processor 52. The processor 52 may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method may be performed by integrated logic circuitry in hardware in processor 52 or by instructions in the form of software. The processor 52 may be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or may implement or perform the methods, steps, and logic blocks disclosed in embodiments of the present invention. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present invention may be embodied directly in the execution of a hardware decoding processor, or in the execution of a combination of hardware and software modules in a decoding processor. The software modules may be located in a random access memory, flash memory, read only memory, programmable read only memory, or electrically erasable programmable memory, registers, etc. as well known in the art. The storage medium is located in a memory 54 and the processor 52 reads the information in the memory 54 and, in combination with its hardware, performs the steps of the method described above.

It is to be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or a combination thereof. For a hardware implementation, the processing units may be implemented within one or more Application Specific Integrated Circuits (ASICs), digital Signal Processors (DSPs), digital Signal Processing Devices (DSPDs), programmable Logic Devices (PLDs), field Programmable Gate Arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described herein, or a combination thereof.

For a software implementation, the techniques described herein may be implemented with modules (e.g., procedures, functions, and so on) that perform the functions described herein. The software codes may be stored in a memory and executed by a processor. The memory may be implemented within the processor or external to the processor.

Further, the processor 52 is specifically configured to monitor the voice signal of the target object for a duration indicated by the first instruction; and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

Further, the processor 52 is specifically configured to receive a repeat play command sent by the server, and select a voice file corresponding to the first command from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

Further, the processor 52 is specifically configured to perform at least one of:

and receiving the voice file sent by the server and storing the voice file.

In some embodiments, the processor 52, when calling and executing the application programs and data stored in the memory 54, specifically performs the following steps: receiving detection data sent by a radar module, wherein the radar module comprises a plurality of radars, and the detection data comprises the state of a target object, the number of the radars for acquiring the data and the position information of the target object; determining a radar corresponding to first detection data, wherein the state of a target object in the first detection data is falling; a first instruction is sent to the radar, and the radar is instructed to select a voice instruction corresponding to the first instruction from a plurality of prestored voice files to play; and receiving a voice signal sent by the radar, and determining whether the target object falls down according to the voice signal.

Further, the processor 52 is specifically configured to decode the voice signal to obtain information including a radar number, an identifier of the first instruction, a number of voice data, and a content of the voice data; if the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

Further, the processor 52 is specifically configured to receive a personalized voice setting instruction sent by the mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

Example IV

The embodiment of the invention also provides a computer readable storage medium storing a computer program, which when being executed by a processor, causes the processor to execute the steps of:

detecting a target object in a building to obtain detection data, wherein the detection data comprises the state of the target object, the number of a radar for acquiring data and the position information of the target object;

transmitting the detection data to a server;

Alternatively, the computer readable storage medium stores a computer program which, when executed by a processor, causes the processor to perform the steps of:

Further, the computer program, when executed by a processor, further causes the processor to perform the steps of: monitoring a voice signal of the target object within the duration indicated by the first instruction; and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

Further, the computer program, when executed by a processor, further causes the processor to perform the steps of: receiving a repeated playing instruction sent by the server, and selecting a voice file corresponding to the first instruction from a plurality of prestored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

Further, the computer program, when executed by a processor, further causes the processor to perform at least one of:

and receiving the voice file sent by the server and storing the voice file.

Further, the computer program, when executed by a processor, further causes the processor to perform the steps of: decoding the voice signal to obtain information comprising a radar number, an identification of the first instruction, a number of voice data and content of the voice data; if the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

Further, the computer program, when executed by a processor, further causes the processor to perform the steps of: receiving a personalized voice setting instruction sent by a mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

The foregoing is a preferred embodiment of the present invention and it should be noted that modifications and adaptations to those skilled in the art may be made without departing from the principles of the present invention and are intended to be comprehended within the scope of the present invention.

Claims

1. A fall detection method, characterized by being applied to a radar module, the radar module comprising a plurality of radars, the plurality of radars comprising a side-mounted radar and a top-mounted radar, the side-mounted radar being mounted on a wall of a building, the top-mounted radar being mounted on a top of the building, the method comprising:

Transmitting the detection data to a server;

2. A fall detection method as claimed in claim 1, wherein the receiving the speech signal returned by the target object and transmitting the speech signal to the server comprises:

3. A fall detection method as claimed in claim 2, wherein, if the content of the speech data is empty, after the step of receiving the speech signal returned by the target object and sending the speech signal to the server, the method further comprises:

4. A fall detection method as claimed in claim 2, wherein the method further comprises at least one of:

and receiving the voice file sent by the server and storing the voice file.

5. A fall detection method, applied to a server, comprising:

6. A fall detection method as claimed in claim 5, wherein determining whether the target object has fallen from the speech signal comprises:

7. A fall detection method as claimed in claim 5, wherein the method further comprises:

receiving a personalized voice setting instruction sent by a mobile terminal;

8. A fall detection method as claimed in claim 5, further comprising at least one of:

9. Fall detection device, its characterized in that is applied to radar module, radar module includes a plurality of radars, a plurality of radars include side dress radar and top dress radar, side dress radar installs on the wall of building, top dress radar installs the top of building, the device includes:

10. Fall detection apparatus according to claim 9, wherein the communication module is specifically configured to monitor the target object for a voice signal during a time period indicated by the first instruction; and encoding and compressing the voice signal, packaging and transmitting the voice signal to the server, wherein the information transmitted to the server comprises a radar number, the identification of the first instruction, the number of the voice data and the content of the voice data.

11. The fall detection apparatus according to claim 10, wherein the communication module is further configured to receive a replay instruction sent by the server, and select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and receiving the voice signal returned by the target object and sending the voice signal to the server.

12. The fall detection apparatus according to claim 10, wherein the communication module is further configured to perform at least one of:

and receiving the voice file sent by the server and storing the voice file.

13. A fall detection apparatus, for use with a server, comprising:

14. Fall detection apparatus according to claim 13, wherein the processing module is specifically configured to decode the speech signal to obtain information comprising a radar number, an identification of the first instruction, a number of speech data and a content of the speech data; if the content of the voice data is empty, sending a repeated playing instruction to the radar, and indicating to select a voice file corresponding to the first instruction from a plurality of pre-stored voice files to play; and if the content of the voice data is not empty, determining whether the target object falls down according to the content of the voice data.

15. A fall detection apparatus as claimed in claim 13, wherein the communication module is further configured to receive a personalized speech setting instruction sent by the mobile terminal; and sending a voice file corresponding to the personalized voice setting instruction to the radar module.

16. The fall detection apparatus according to claim 13, wherein the communication module is further configured to perform at least one of: