CN112637418A - Danger reminding method and device - Google Patents

Abstract

The invention discloses a danger reminding method and a device, wherein the method comprises the following steps: when it is determined that the user is using the immersive application software and/or that a target condition is within a preset target condition range, initiating hazard monitoring, the target condition comprising time, place, or weather; when a danger source is monitored, acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger source characteristic library to obtain a danger characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding the user. Therefore, the user can find danger in time and avoid accidents.

Description

Danger reminding method and device

Technical Field

The invention relates to the field of mobile terminals, in particular to a danger reminding method and device.

Background

With the rapid development of intelligent terminals and mobile technologies, various mobile terminals such as smart phones and tablet computers have become indispensable portable devices in people's lives. Applications in mobile terminals are also constantly evolving, such as news information, social applications, game applications, etc.

There are many immersive application software in mobile terminals and when a user is using such application software, attention is focused entirely on the mobile terminal. If the user is in danger at this time, the user may not pay attention to the danger, so that the user is exposed to the danger and accidents are easily caused.

For example, when a user walks, the user cannot find dangers such as nearby high-speed vehicles, ground obstacles, high-altitude falling objects and the like in time when using immersive applications such as webpages, videos or games through a mobile phone. Thereby exposing the user to the danger and causing an accident.

Disclosure of Invention

The embodiment of the invention provides a danger reminding method and a device, wherein the method comprises the following steps: when it is determined that the user is using the immersive application software and/or that a target condition is within a preset target condition range, initiating hazard monitoring, the target condition comprising time, place, or weather; when a danger source is monitored, acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger source characteristic library to obtain a danger characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding the user. Therefore, the user can find danger in time and avoid accidents.

Optionally, with reference to the first aspect, in a first possible implementation manner of the first aspect, the determining that the user is using the immersive application software specifically includes: and when at least one of continuous audio output within the first preset time length, continuous screen lightening within the second preset time length and receiving user input within the third preset time length is greater than or equal to a preset input time threshold is detected, determining that the user is using the immersive application software. This may enable hazard monitoring by detecting audio constant output, constant bright screen, or user constant input to determine that the user is using the immersive application software.

Optionally, with reference to the first aspect, in a second possible implementation manner of the first aspect, after acquiring the feature of the hazard source and comparing the feature of the hazard source with a preset hazard source feature library to obtain a hazard feature value, the method further includes: determining the danger level of the danger source according to the characteristics of the danger source; when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding a user comprises the following steps: and when the dangerous characteristic value exceeds a preset characteristic value threshold and the dangerous level of the dangerous source exceeds a preset dangerous level, reminding a user. When the danger level of the danger source exceeds the preset danger level, the user is reminded, so that false alarms can be reduced, and the user experience is improved.

Optionally, with reference to the first aspect, in a third possible implementation manner of the first aspect, the characteristics of the hazard source include: the system comprises a dangerous source profile, dangerous source and environment background contrast, dangerous source mobility, dangerous source darkness, dangerous source speed, dangerous source distance, dangerous source volume and dangerous source frequency.

Optionally, with reference to the first aspect, in a fourth possible implementation manner of the first aspect, when the dangerous feature value exceeds the preset feature value threshold, the reminding the user includes: when the dangerous characteristic value exceeds the preset characteristic value threshold, reminding the user according to a preset reminding mode, wherein the preset reminding mode comprises the following steps: displaying screen occupation warning color, text prompt, voice prompt, vibration prompt and remote prompt. Reminding the user through the preset reminding mode can ensure that the user can be interrupted to use the immersive application software, and can give an alarm to the user in time.

Optionally, with reference to the first aspect, in a fifth possible implementation manner of the first aspect, when at least one dangerous feature value exceeds a preset feature value threshold, the method further includes: a method for prompting a user to avoid danger comprises voice prompt, text prompt and light prompt. By providing danger avoiding method prompt for the user, the user can be prevented from getting rid of the danger when encountering the danger, and the danger can be avoided in time according to the danger prompting method.

Optionally, with reference to the first aspect, in a sixth possible implementation manner of the first aspect, when a feature that is not included in the preset hazard source feature library is acquired, the feature that is not included in the preset hazard source feature library is sent to a cloud, and the feature that is not included in the preset hazard source feature library is used for the cloud to update the preset hazard source feature library. Therefore, the preset danger characteristic library can be continuously updated, so that the identification of the danger source is more accurate.

A second aspect of the embodiments of the present invention provides a danger warning device, which is characterized in that the device includes: a processing module for initiating hazard monitoring when it is determined that the user is using the immersive application software and/or that a target condition is within a preset target condition range, the target condition including time, place or weather; the monitoring module is used for collecting the characteristics of the hazard source when the hazard source is monitored; the processing module is used for comparing the characteristics of the hazard source with a preset hazard source characteristic library to obtain a hazard characteristic value; and the reminding module is used for reminding the user when the processing module determines that the characteristic value of the hazard source exceeds the preset characteristic value threshold. Therefore, the user can find danger in time and avoid accidents.

Optionally, with reference to the second aspect, in a first possible implementation manner of the second aspect, the processing module is further configured to determine that the user is using the immersive application software when at least one of a continuous audio output within the first preset time period, a continuous screen-on within the second preset time period, and a number of times of receiving the user input within the third preset time period is greater than or equal to a preset input number threshold is detected.

Optionally, with reference to the second aspect, in a second possible implementation manner of the second aspect, the processing module is further configured to determine a risk level of the hazard source according to a feature of the hazard source; the reminding module is also used for reminding the user when the dangerous characteristic value exceeds a preset characteristic value threshold and the dangerous level of the dangerous source exceeds a preset dangerous level.

Optionally, with reference to the second aspect, in a third possible implementation manner of the second aspect, the processing module is further configured to remind the user according to a preset reminding manner when the processing module determines that the dangerous feature value exceeds the preset feature value threshold, where the preset reminding manner includes: displaying screen occupation warning color, text prompt, voice prompt, vibration prompt and remote prompt.

Optionally, with reference to the second aspect, in a fourth possible implementation manner of the second aspect, the processing module is further configured to prompt a user with a method for avoiding a danger, where the method includes a voice prompt, a text prompt, and a light prompt.

Optionally, with reference to the second aspect, in a fifth possible implementation manner of the second aspect, the apparatus further includes a sending module, configured to send, to a cloud, a feature that is not present in the preset hazard source feature library when the monitoring module acquires the feature that is not present in the preset hazard source feature library, where the feature that is not present in the preset hazard source feature library is used for the cloud to update the preset hazard source feature library. Therefore, the preset danger characteristic library can be continuously updated, so that the identification of the danger source is more accurate.

A third aspect of the embodiments of the present invention provides a danger warning device, which is characterized in that the device includes: the system comprises a low-power consumption ultra-wide angle sensor, an ultrasonic sensor and an embedded neural network processor NPU, wherein the low-power consumption ultra-wide angle monitoring sensor comprises an ultra-wide angle lens, the NPU is used for starting the low-power consumption ultra-wide angle sensor and the ultrasonic sensor when a user uses immersive application software and/or a target condition is within a preset target condition range, the low-power consumption ultra-wide angle sensor and the ultrasonic sensor are used for monitoring dangers, and the target condition comprises time, place or weather; when the low-power consumption ultra-wide-angle sensor and/or the ultrasonic sensor monitor the dangerous source, the low-power consumption ultra-wide-angle sensor and the ultrasonic sensor collect the characteristics of the dangerous source, and the NPU compares the characteristics of the dangerous source with a preset dangerous source characteristic library to obtain a dangerous characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, the NPU reminds the user. Therefore, the user can find danger in time and avoid accidents.

The invention provides a danger reminding method and a device, and the method can comprise the following steps: when it is determined that the user is using the immersive application software and/or that a target condition is within a preset target condition range, initiating hazard monitoring, the target condition comprising time, place, or weather; when a danger source is monitored, acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger source characteristic library to obtain a danger characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding the user. Therefore, the user can find danger in time and avoid accidents. Meanwhile, the scheme judges based on the characteristic value, so that the judgment accuracy can be improved.

Drawings

Fig. 1 is a schematic view of a dangerous scene according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of another dangerous scene provided in the embodiment of the present invention;

FIG. 3 is a schematic diagram of another dangerous scene provided in the embodiment of the present invention;

FIG. 4 is a schematic diagram of an embodiment of a danger-reminding method according to an embodiment of the present invention;

fig. 5 is a flowchart of a danger notification method according to an embodiment of the present invention;

fig. 6 is a schematic diagram of an embodiment of a danger-reminding device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

The term "and/or" appearing in the present application may be an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone. In addition, the character "/" in this application generally indicates that the former and latter related objects are in an "or" relationship.

The terms "first," "second," and the like in the description and in the claims of the present application and in the above-described drawings are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It will be appreciated that the data so used may be interchanged under appropriate circumstances such that the embodiments described herein may be practiced otherwise than as specifically illustrated or described herein. Moreover, the terms "comprises," "comprising," and any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but may include other steps or modules not expressly listed or inherent to such process, method, article, or apparatus.

With the rapid development of intelligent terminals and mobile technologies, various mobile terminals such as smart phones and tablet computers have become indispensable portable devices in people's lives. Applications in mobile terminals are also constantly evolving, such as news information, social applications, game applications, etc. There are many immersive application software in mobile terminals and when a user is using such application software, attention is focused entirely on the mobile terminal. If the user is in danger at this time, the user may not pay attention to the danger, so that the user is exposed to the danger and accidents are easily caused.

For example, when the user walks, the user cannot find dangers such as a nearby high-speed vehicle (as shown in fig. 1), a ground obstacle (as shown in fig. 2), a high-altitude falling object (as shown in fig. 3), and the like in time by using an immersive application such as a web page, a video, or a game through a mobile phone. Thereby exposing the user to the danger and causing an accident. When the user uses the immersive application software, the attention is focused on the mobile terminal, and if the user encounters danger, the user may not pay attention to the danger, so that the user is exposed to the danger, and accidents are caused.

The traditional danger reminding method needs to start a monitoring program, is high in complexity, and cannot timely monitor dangers for terminal equipment which does not download installation software in advance. In addition, the traditional danger reminding method needs the terminal equipment to shoot a complete high-definition image or start a microphone to monitor risks, so that the power consumption of the terminal equipment is high. When a user is using a terminal device (such as a mobile phone), the rear camera of the terminal device faces the ground, and the front camera faces the user and may be blocked by a human body. The warning mode may not be timely or interrupt the use of the immersive application of the user, and a danger avoidance method prompt is lacked, so that the warning is untimely and ambiguous, and even if the prompt is given, the user is highly likely to be caused to be free from the warning, and the risk cannot be avoided in time. Risks cannot be classified, excessive alarms can affect user experience, and real high-level risks cannot be handled in the first time when multiple risk sources exist in the environment. Meanwhile, the power consumption of the mobile phone camera is large, the mobile phone camera cannot be in an always-on state, and a large amount of redundant information exists.

The invention provides a danger reminding method which can monitor danger when a user uses immersive application software and remind the user when the danger is determined to be found, so that safety accidents of the user can be avoided. As shown in fig. 4, the method includes:

101. the hazard monitoring is initiated when it is determined that the user is using the immersive application software and/or that the target condition is within a preset target condition range.

The danger monitoring is initiated when the mobile terminal determines that the user is using the immersive application software and/or that the target condition is within a preset target condition range. The manner in which the terminal device determines that the user is using the immersive application software may include, but is not limited to, the following:

1. detecting whether continuous audio output exists within a first preset time period, wherein the audio output can comprise wired audio output, wireless audio output and loudspeaker audio output. Illustratively, the first predetermined period of time is 30 seconds. When there is a continuous audio output for the first period of time, the mobile terminal may determine that the user is using the immersive application software.

2. And detecting whether the screen is continuously on within a second preset time, wherein the second preset time can be preset screen-off time of the mobile phone, or can be user-defined screen-on duration or a shorter duration of the preset screen-off time of the mobile phone and the user-defined screen-on duration. Illustratively, if the preset screen-off duration of the mobile phone is 2 minutes, the user-defined bright screen duration is 5 minutes, and the second preset duration may be 2 minutes, which is the smaller value of the preset screen-off duration of the mobile phone and the user-defined bright screen duration. If the terminal device detects a continuous bright screen state within 2 minutes, it can be determined that the user is using the immersive application software.

3. Detecting that the number of times of receiving the user input within the third preset time period is greater than or equal to a preset input number threshold, for example, the third preset time period may be 10 seconds, and the preset input number threshold may be 5 times. When the terminal device detects that the number of times of user input is greater than or equal to 5 times in 10 seconds, it may be determined that the user is using the immersive application software. The user input may include the number of times the user has pressed a key or touched the screen.

It should be noted that the terminal device may determine whether the user is using the immersive application software jointly according to one or several items described above. For example, when the terminal device detects continuous audio output for a first preset time period and continuous bright screen for a second preset time period, the terminal device may determine that the user is using the immersive application software.

The danger monitoring is initiated when the terminal device determines that the user is using the immersive application software. This terminal equipment starts dangerous control and includes: and starting the low-power consumption ultra-wide angle monitoring sensor and the ultrasonic sensor. This super wide angle of low-power consumption monitoring sensor can adopt super wide angle camera lens, super low-power consumption image sensor, and the collection angle can reach 180 degrees to 360 degrees, can cooperate the distortion to correct recognition algorithm and only detect the power saving image monitoring algorithm of eigenvalue. The defect that the wide-angle monitoring can be realized only by rotating the current image acquisition sensor can be avoided, moving parts can be reduced, and the stability of the framework is improved. The ultra-low power consumption wide-angle sensor can be a frog-eye sensor, and is called as a frog-eye sensor below.

This terminal equipment can be according to the image that this frog-eye sensor gathered, 4 characteristics of dangerous source profile, dangerous source and environment background contrast, dangerous source mobility, dangerous source darkness can be drawed in cooperation electron frog-eye algorithm, and the judgement based on the characteristic can promote the identification accuracy, and compressed image data volume can simplify the judgement of dangerous source, filters the interference source in the environment background, and simultaneously, the image sensor who uses lower pixel also can practice thrift the consumption.

The ultrasonic sensor can emit ultrasonic waves to a specific direction for detection, and realizes the speed of a dangerous source, the distance of the dangerous source and the volume of the dangerous source by matching with a specific algorithm. Therefore, the interference of sound sources in the environment can be avoided while the danger early warning is realized.

It should be noted that, the user may also set some target conditions in a customized manner, and may also start the risk monitoring when the target conditions are met. The target conditions include, but are not limited to: time, place, or weather, etc.

For example, the user may locally establish a dangerous spot database in the terminal device, and if the target condition is a spot, the range of the preset target condition may be the dangerous spot database, and the dangerous spot database may include: dangerous place information or place types such as traffic intersections, environments higher than the flat ground by more than 2 meters, the front of an elevator car, a water area, specific longitude and latitude and the like. When the location where the user is located at a location or a location type in the hazardous location database, the terminal device may determine that the target condition is within a preset target condition range.

If the target condition is time, the preset target condition may be a preset time range, and the preset time range may be: the terminal device may acquire the local sunset time and sunrise time by starting to stopping months, starting to stopping dates, starting to stopping times, or sunset to sunrise. When the current time is within the preset time range, the terminal device may determine that the target condition is within the preset target condition range.

The user can also set specific weather, the range of the preset target condition can be a preset weather condition range, and the preset weather condition can be as follows: rain, snow, fog, etc. The terminal equipment can acquire real-time weather conditions through a network or a sensor. When the current weather is one of the preset weathers, the terminal device may determine that the target condition is within a preset target condition range.

The target condition may also include one or more of time, place, or weather. For example, the terminal device may determine that the target condition is within the preset target condition range when it is determined that the current time is within the preset time range and the current weather is within the preset weather condition range. This allows monitoring of the hazard at a particular time, location or weather condition. For example, when the elderly or the children go out at night or pass through a busy road, monitoring is performed, so that danger can be prevented.

The target condition may also be a characteristic of the terminal device itself, for example, the target condition may be considered to be within a preset target condition range when the screen is turned off.

The terminal device may initiate hazard monitoring when the user is using the immersive application software and/or the target condition is within a preset target condition range.

102. And when the danger source is monitored, acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger characteristic library to obtain a danger characteristic value.

When the terminal equipment monitors the dangerous source, the characteristics of the dangerous source are collected and compared with a preset dangerous characteristic library to obtain a dangerous characteristic value. The hazard source may be characterized according to the characteristics acquired by a frog-eye sensor and an ultrasonic sensor.

This terminal equipment can be according to the image that this frog-eye sensor gathered, 4 characteristics of dangerous source profile, dangerous source and environment background contrast, dangerous source mobility, dangerous source darkness can be gathered to cooperation electron frog-eye algorithm, and the judgement based on the characteristic can promote the identification accuracy, and compressed image data volume can simplify the judgement of dangerous source, filters the interference source in the environment background, and simultaneously, the image sensor who uses lower pixel also can practice thrift the consumption.

After the frog-eye sensor collects the image of the hazard source, the image is transmitted to an embedded neural Network Processing Unit (NPU), and the NPU can correct the image through a distortion correction recognition algorithm or a barrel distortion correction recognition algorithm, so that distortion can be avoided.

The ultrasonic sensor can emit ultrasonic waves to a specific direction for detection, and realizes the speed of a dangerous source, the distance of the dangerous source and the volume of the dangerous source by matching with a specific algorithm. Therefore, the interference of sound sources in the environment can be avoided while the danger early warning is realized.

The characteristics of the hazard source may include, but are not limited to:

1. hazard source profile.

The NPU extracts a hazard source profile, which may be composed of lines, and illustratively, a polygon, from the corrected image acquired by the frog-eye sensor.

2. Hazard source and environmental background contrast.

The NPU can acquire the contrast between the dangerous source and the background environment according to the corrected image, the contrast is used for distinguishing a foreground and a background, the foreground is the dangerous source, and the background is the environment where the dangerous source is located. Therefore, redundant background data can be filtered, data are simplified, and accuracy and timeliness of danger identification are improved. The hazard may be brighter or darker than the environment, and may be filtered out of the background as long as there is contrast.

3. The mobility of the hazard source.

The terminal equipment can also detect the moving state of the dangerous source, and can only extract moving objects in the image as the monitored dangerous source, such as moving vehicles and the like. This allows locking only the moving hazard source, further simplifying the data. The manner in which the terminal device determines the movement state of the hazard source is not limited. Illustratively, the terminal device may continuously capture a plurality of images by the frog-eye sensor, and the NPU determines whether the object moves by determining whether the relative position of the object in the plurality of images changes. Thus, stationary objects can be filtered out and only moving objects can be detected.

4. Darkness of the hazard source.

The terminal equipment can detect the brightness change of the hazard source, and the position of the hazard source is determined by detecting the brightness change of the hazard source entering the detection field of view in the image and detecting the position of the hazard source in the image. When the photographed hazard source is at the center position of the image, the terminal device may determine that the hazard source is facing the user. When the photographed hazard source is at an edge position of the image, the terminal device may determine that the hazard source is at a side position of the user. For example, if a car is captured at the center of the image, indicating that the car is facing the user, and if the car is at the edge of the image, the car can be determined to be at the side of the user.

5. The hazard source velocity.

The terminal equipment can also judge the moving speed of the hazard source according to the ultrasonic probe of the ultrasonic sensor. Illustratively, the terminal device periodically transmits ultrasonic waves to the same dangerous source through the ultrasonic probe, receives an echo, calculates a distance difference between two detections through echo return time, and divides the distance difference by a detection period, thereby obtaining the moving speed of the dangerous source. For example, the ultrasonic probe periodically transmits a detection ultrasonic wave to the hazard source for 100 milliseconds, and if the hazard source is detected to be 0.5 m away from the user by two detections, the moving speed of the hazard source is determined to be 5 m/s.

6. The distance of the source of danger.

The terminal equipment can judge the distance between the danger source and the user according to the ultrasonic probe. For example, the terminal device may obtain the distance between the hazard source and the user by multiplying the time difference between the transmission of the ultrasonic wave and the reception of the echo wave by the speed of the ultrasonic wave propagating in the air.

7. The source of danger is volumetric.

The terminal equipment can judge the volume of the hazard source through the ultrasonic probe. Illustratively, the volume of the hazard source can be detected by detecting the length, width, and height of the hazard source.

8. The frequency of the hazard source.

The terminal device can detect the frequency of the hazard source through the ultrasonic probe. For example, the frequency of the vehicle may be detected. The higher the frequency of the vehicle, the higher the degree of danger, indicating that the vehicle may be accelerating.

After the terminal device acquires the characteristics of the hazard source, the terminal device may compare the acquired characteristics of the hazard source with a preset hazard source characteristic library to obtain a hazard characteristic value. The preset hazard source feature library includes, but is not limited to, the features of the hazard sources. The terminal device may determine the dangerous feature value according to one or more of the features of the dangerous source, and the determination mode may be set by a user or preset by the system. Illustratively, when the terminal device determines that the distance between the hazard source and the user is less than 10 meters, if the speed of the hazard source is 0.5 meters per second, the hazard characteristic value is determined to be 50; if the speed of the hazard source is 0.8 meters per second, determining that the hazard characteristic value is 80; and if the speed of the hazard source is greater than or equal to 1 meter per second, determining the hazard characteristic value to be 100. The preset characteristic value threshold may be 80, and when the preset characteristic value threshold is exceeded, the user is reminded.

It should be noted that the detection range of the ultrasonic probe can be set through power control, so that dangerous false alarms over a long distance can be reduced, and power consumption can be reduced. Illustratively, the ultrasound may be made to detect sources of danger only within a 10 meter range, subject to power limitations. Sources of danger beyond the 10 meter range are not detectable due to the limited transmit power. Therefore, the dangerous source is far away from the user, the warning can be avoided, and meanwhile, the transmitting power can be reduced. The direction of detection by the ultrasonic probe can also be set. For example, if the user walks close to a roadside railing without danger and the car is close to a lane, the ultrasonic probe only detects the left lane.

Optionally, when the risk is monitored, the risk source may be evaluated according to a risk level evaluated by an international risk evaluation algorithm preset in the terminal device. The risk assessment algorithm may be, but is not limited to, an lec (lelellihood exposure sequence) algorithm. The LEC algorithm may be based on: the likelihood of an accident (likelihood), the frequency of exposure of personnel to a hazardous environment (exposure), and the consequences that may result once an accident has occurred (consequence) are evaluated in three dimensions. And determining different scores according to different grades of the three factors, and evaluating the risk by multiplying the three scores. The risk score D ═ lxe × C.

Table 1 below is a likelihood, the probability of a hazard occurring and the corresponding score value.

TABLE 1

Table 2 below is exposure, the frequency with which personnel are exposed to hazardous environments.

TABLE 2

Score value	Degree of frequency
		10	Continuous exposure
6	Exposure during daily working hours
		3	Once a week, or by accidental exposure
2	Once monthly exposure
		1	Several exposures per year
0.5	Very rare exposures

The following table 3 shows the consequences of an accident:

TABLE 3

Score value	Consequence of
		100	Big disaster
40	Disaster
		15	Is very serious
7	Severe severity of disease
		3	Of great importance
1	Is eye-catching, in general

And after the terminal equipment evaluates from the three dimensions according to the collected characteristics of the hazard source, obtaining a risk score according to the scores of the three dimensions, and comparing the risk score with the standard risk score to obtain the risk degree and the risk grade.

TABLE 4

D value	Degree of risk	Risk rating
			>320	Is extremely dangerous	A
160-320	High risk	B
			70-160	Significant danger	C
20-70	General danger	D
			<20	Slight danger	E

Optionally, after the danger is monitored, the danger source may be classified, and the danger source may be further sub-classified while being classified. For example, sources of danger can be classified as: motor vehicles, hazardous organisms, building related hazards, and the like. The motor vehicles can be further divided into subclasses such as large-sized vehicles, medium-sized vehicles and small-sized vehicles; the dangerous organisms can be further divided into the subclasses of beasts, large-scale livestock, poisonous insects, pets and the like; the related dangers of the building can be further divided into high-altitude falling objects of a dangerous building, steep stairs in the building, non-arrival of an elevator at a door and other subclasses.

Optionally, the hazard sources may be classified according to the degree of hazard or the occurrence probability. For example, according to the LEC algorithm described above, the hazard sources may be divided into five levels A-E. It may be arranged to alert when the risk level exceeds a preset risk level. For example, when the risk level exceeds the level C, the warning is performed, so that the threshold of warning is increased, and when multiple risk sources occur in the environment, the real high-level risk can be handled in the first time, so that the user experience can be improved. The terminal equipment can remind a user when the dangerous characteristic value exceeds the preset characteristic value threshold and the dangerous level of the dangerous source exceeds the preset dangerous level.

It should be noted that the user may also set the time, place or type of hazard source that is not monitored and not reminded. For example, the user may select a safe time period or a safe area, and when the current time is the safe time period or the location of the user is located in the safe area, the user is not monitored and reminded. When an unmonitored hazard source type is identified, it is unmonitored and not alerted. Therefore, the initiative of the supervisor of the user can be exerted, and the user experience is improved.

103. And when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding the user.

As shown in step 102, when the dangerous feature value exceeds the preset feature value threshold, the user is reminded. The method for reminding the user by the terminal device includes but is not limited to: the method comprises the steps of screen occupation warning color display, voice prompt, vibration prompt and remote prompt.

It should be noted that, while the above-mentioned danger warning method is used, the terminal device may also prompt the user with a danger avoiding method according to a locally stored reference method for avoiding danger, and the danger avoiding method may include, but is not limited to, the following forms:

1. and displaying the screen occupation warning color.

When the dangerous characteristic value exceeds the preset characteristic value threshold, a screen occupation warning color is displayed on the screen of the terminal equipment, and the area occupied by the screen occupation warning color can be set by a user in a self-defined mode. Illustratively, the screen-occupying warning color may occupy the entire screen or a user-defined portion of the screen. The color of the screen occupation alarm can be customized by a user. It can be distinguished according to the type of hazard. Illustratively, red may default to high risk, yellow to medium risk, and blue to low risk. Dangerous sources which threaten personal safety such as vehicles, falling objects, elevators and the like and do not reach the door are set to be red, financial dangers are yellow, and information safety threats are blue. All colors are customizable by the user.

2. And (6) text prompting.

When the screen occupation warning color is displayed, prompting can be carried out through characters, warning information is displayed through a specific font, and a method for avoiding risks is prompted to a user. For example, in a conspicuous position of the screen, a large icon and a large font display reference method for avoiding risks, if a user walks near the left side and there is a risk of collision of an incoming vehicle on the right side, a left arrow is displayed, and the user is prompted with characters: "right side coming, left side evading".

3. And (6) voice prompt.

And reminding is carried out through voice. When detecting that the earphone is connected with the terminal equipment, the earphone can be adopted for voice reminding, and if the earphone is not connected with the terminal equipment, the loudspeaker can be adopted for voice reminding. The content of the voice prompt may include key information such as the type, location, and distance of the hazard source.

The voice reminding method can inform a user of the evasive reference method through voice while reminding danger. For example, if the user walks near the left side and the right side is in danger of collision with an incoming vehicle, a message that the incoming vehicle is left-avoided is broadcasted through voice.

4. And (5) vibration prompting.

And (4) prompting danger early warning through motor vibration in different directions. For example, the mobile phone vibrates in a left-right double-motor mode. When the left side of the user is dangerous, the right motor vibrates to prompt the user to avoid the danger to the right.

5. And lighting for prompting.

The mobile phone flash lamp is actively lightened to warn. Illustratively, a dark environment lights up a flashlight, alerts vehicles to or from, etc., or assists the user in illuminating.

6. And (6) remote prompting.

The emergency contact set by the user can remotely alarm. If the terminal device monitors that the infants in home run danger, the warning signal can be sent out and simultaneously a warning signal can be sent to the preset emergency contact relatives and friends.

It can be understood that when the dangerous characteristic value exceeds the preset characteristic value threshold, the terminal device may perform a prompt through one or more of the above items. For example, the voice prompt and the text prompt may be simultaneously performed, and are not limited herein.

It should be noted that, when the terminal device acquires a feature that is not present in the hazard source feature library, the terminal device may also send the feature that is not present in the preset hazard source feature library to the cloud, the cloud optimizes an existing hazard source feature value or adds a new feature value through algorithms such as dimension reduction, clustering, classification, and the like, and then sends the new feature value to the terminal device, so that the terminal device updates the local hazard source feature library.

Referring to fig. 5, a flowchart of a danger notification method is further provided in an embodiment of the present application, where the flowchart may include: user Interface (UI) settings: the method comprises the steps of early warning and reminding preference, real-time detection of a danger source, a danger source feature library and Artificial Intelligence (AI) identification and early warning of a user.

The UI settings can be set by the user, and may include: function trigger conditions, monitoring/alarm filtering, and alarm modes. Wherein the function triggering conditions include: time trigger, place trigger, weather trigger, and user behavior trigger, which may include earphone connection, bright screen application, user key input, please refer to the way in which the terminal device determines that the user is using the immersive application software in step 101 and some target conditions set by the user in a customized manner. In addition, the user can also set the AI to take over automatically. The monitoring/alarm filtering comprises that monitoring is not started under specified conditions, and alarm is not started under specified conditions. The user may also set an alert mode, which may refer to step 103, and a reference of whether to enable a hazard avoidance mode.

The source of danger can be monitored in real time as it is determined that the user is using the immersive application, with feature recognition from video and audio dimensions by electronic frowneye sensors as well as ultrasonic sensors. The electronic frog-eye sensor can identify the type of a hazard source, and the ultrasonic sensor can identify the distance, the volume and the speed of the hazard source. The electronic frog-eye sensor comprises a fisheye ultra-wide-angle lens, and the visual range can be 180-360 degrees. The acquired images are corrected through a barrel distortion correction recognition algorithm, then the images are transmitted to a processor through an ultra-low power consumption image sensor (the power consumption is 700 microwatts), the dangerous source is recognized through a dynamic object frog-eye calculation recognition method, and the characteristics of the dangerous source are extracted and output according to the characteristics of the dangerous source images. The ultrasonic sensor comprises an ultrasonic transmitting and receiving device, and the distance, the volume and the speed between the hazard source and the user are measured by an estimation algorithm of the distance, the volume and the speed between the transmitting and receiving ultrasonic waves and the object.

A danger source characteristic library is built in the terminal equipment and can be identified through AI, and the identified danger source characteristics comprise image characteristics and audio characteristics. And classifying and grading the hazard sources through a grading algorithm, wherein the grading algorithm comprises clustering, classification and regression. The AI of local and high in the clouds of the terminal equipment is used for controlling, and the danger source can be identified based on the characteristics. And when the danger source is identified, generating an early warning and avoiding mode aiming at the danger source. Monitoring, alarm start stop and filtering management. And collecting, updating and pushing new features to the cloud.

And when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding the user. Including interrupting the use of immersive applications by the user and referencing danger avoidance approaches to early warning.

The embodiment of the invention provides a danger reminding method, which can be applied to terminal equipment, wherein the terminal equipment can comprise a frogman eye sensor, an ultrasonic sensor and an embedded neural network processor NPU, the NPU is used for starting the frogman eye sensor and the ultrasonic sensor when a user uses immersive application software or a target condition is in a preset condition range, the frogman eye sensor and the ultrasonic sensor are used for monitoring dangers, and the target condition comprises time, place or weather; when the frog eye sensor and the ultrasonic sensor monitor the dangerous source, the frog eye sensor and the ultrasonic sensor acquire the characteristics of the dangerous source, and the NPU compares the characteristics of the dangerous source with a preset dangerous source characteristic library to obtain a dangerous characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, the NPU reminds the user. Therefore, when danger is met, the user can be reminded in time, and the user is prevented from being exposed to the danger.

A second embodiment of the present invention provides a danger alerting device 20, where the danger alerting device 20 is configured to perform a danger alerting method provided in the first embodiment of the present invention, as shown in fig. 6, the danger alerting device 20 includes:

a processing module 201 for starting danger monitoring when it is determined that the user is using the immersive application software or that the target condition is within a preset target condition range, the target condition including time, place or weather. Please refer to step 101 of the first embodiment for understanding, which is not described herein again.

The processing module 201 is further configured to compare the characteristic of the hazard source with a preset hazard source characteristic library to obtain a hazard characteristic value. Please refer to step 102 of the first embodiment for understanding, which is not described herein again.

The processing module 201 is further configured to determine that the user is using the immersive application software when at least one of a continuous audio output within a first preset time period, a continuous screen-on within a second preset time period, and a number of times of receiving a user input within a third preset time period is greater than or equal to a preset input number threshold is detected. Please refer to step 101 of the first embodiment for understanding, which is not described herein again.

The processing module 201 is further configured to determine that the target condition is within a preset target condition range, where the target condition includes time, place, or weather. Please refer to step 101 of the first embodiment for understanding, which is not described herein again.

This processing module 201 is further configured to, when the processing module determines that the dangerous characteristic value exceeds the preset characteristic value threshold, remind the user according to a preset reminding mode, where the preset reminding mode includes: displaying screen occupation warning color, text prompt, voice prompt, vibration prompt and remote prompt. Please refer to step 103 of the first embodiment for understanding, which is not described herein again.

The processing module 201 is further configured to prompt a user with a method for avoiding danger, where the method includes voice prompt and text prompt. Please refer to step 103 of the first embodiment for understanding, which is not described herein again.

And the monitoring module 202 is used for collecting the characteristics of the danger source when the danger source is monitored. Please refer to step 102 of the first embodiment for understanding, which is not described herein again.

And the reminding module 203 is used for reminding the user when the processing module determines that the characteristic value of the hazard source exceeds the preset characteristic value threshold. Please refer to step 102 and step 103 of the first embodiment for understanding, which are not described herein again.

The reminding module 203 is further configured to remind the user when the danger characteristic exceeds a preset characteristic value threshold and the danger level of the hazard source exceeds a preset danger level. Please refer to step 102 and step 103 of the first embodiment for understanding, which are not described herein again.

A sending module 204, configured to send, to a cloud, a feature that is not present in the preset hazard source feature library when the monitoring module acquires the feature that is not present in the preset hazard source feature library, where the feature that is not present in the preset hazard source feature library is used for the cloud to update the preset hazard source feature library. Please refer to step 103 of the first embodiment for understanding, which is not described herein again.

The method and the device for reminding danger provided by the embodiment of the invention are described in detail, a specific example is applied in the text to explain the principle and the implementation of the invention, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention. Although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (14)

1. A method for danger alerting, the method comprising:

initiating hazard monitoring when it is determined that the user is using the immersive application software and/or that a target condition is within a preset target condition range, the target condition including time, place, or weather;

when a danger source is monitored, acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger source characteristic library to obtain a danger characteristic value;

and when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding a user.

2. The danger alerting method of claim 1, wherein said determining that the user is using the immersive application software specifically comprises:

and when at least one of continuous audio output within a first preset time period, continuous screen lightening within a second preset time period and receiving user input within a third preset time period is detected, determining that the user is using the immersive application software.

3. The danger reminding method according to claim 1, wherein after acquiring the characteristics of the danger source and comparing the characteristics of the danger source with a preset danger source characteristic library to obtain a danger characteristic value, the method further comprises:

determining the danger level of the danger source according to the characteristics of the danger source;

when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding a user comprises the following steps:

and when the dangerous characteristic value exceeds a preset characteristic value threshold and the dangerous level of the dangerous source exceeds a preset dangerous level, reminding a user.

4. The danger alerting method of claim 1, wherein the characteristics of the danger source comprise:

the system comprises a dangerous source profile, dangerous source and environment background contrast, dangerous source mobility, dangerous source darkness, dangerous source speed, dangerous source distance, dangerous source volume and dangerous source frequency.

5. The danger prompting method according to claim 1, wherein when the danger characteristic value exceeds a preset characteristic value threshold, the prompting the user comprises:

when the dangerous characteristic value exceeds a preset characteristic value threshold, reminding a user according to a preset reminding mode, wherein the preset reminding mode comprises the following steps: displaying screen occupation warning color, text prompt, voice prompt, vibration prompt and remote prompt.

6. The danger alerting method of claim 1, wherein when the at least one dangerous characteristic value exceeds a preset characteristic value threshold, the method further comprises:

a method for avoiding danger is prompted to a user, and the method comprises voice prompt, text prompt and light prompt.

7. The danger alerting method of claim 1, further comprising:

and when the features which are not in the preset hazard source feature library are acquired, the features which are not in the preset hazard source feature library are sent to a cloud end, and the features which are not in the preset hazard source feature library are used for updating the preset hazard source feature library by the cloud end.

8. A hazard reminder apparatus, the apparatus comprising:

the system comprises a processing module, a display module and a display module, wherein the processing module is used for starting danger monitoring when determining that the user uses the immersive application software and/or target conditions are within a preset target condition range, and the target conditions comprise time, place or weather;

the monitoring module is used for collecting the characteristics of the danger source when the danger source is monitored;

the processing module is used for comparing the characteristics of the hazard source with a preset hazard source characteristic library to obtain a hazard characteristic value;

and the reminding module is used for reminding the user when the processing module determines that the characteristic value of the hazard source exceeds the preset characteristic value threshold.

9. The danger alerting device of claim 8,

the processing module is further configured to determine that the user is using the immersive application software when at least one of a continuous audio output within a first preset time period, a continuous screen-on within a second preset time period, and a number of times of receiving user input within a third preset time period is greater than or equal to a preset input number threshold is detected.

10. The danger alerting device of claim 8,

the processing module is further used for determining the danger level of the danger source according to the characteristics of the danger source;

the reminding module is further used for reminding the user when the dangerous characteristic value exceeds a preset characteristic value threshold and the dangerous level of the dangerous source exceeds a preset dangerous level.

11. The danger alerting device of claim 8,

the processing module is further configured to remind a user according to a preset reminding mode when the processing module determines that the dangerous characteristic value exceeds a preset characteristic value threshold, where the preset reminding mode includes: displaying screen occupation warning color, text prompt, voice prompt, vibration prompt and remote prompt.

12. The danger alerting device of claim 8,

the processing module is also used for prompting a method for avoiding danger to a user, and the method comprises voice prompt, text prompt and light prompt.

13. The hazard reminder device of claim 8, said device further comprising:

the sending module is used for sending the features which are not in the preset danger source feature library to a cloud end when the monitoring module collects the features which are not in the preset danger source feature library, and the features which are not in the preset danger source feature library are used for updating the preset danger source feature library by the cloud end.

14. A hazard reminder apparatus, the apparatus comprising: a low-power consumption ultra-wide angle monitoring sensor, an ultrasonic sensor and an embedded neural network processor NPU, wherein the low-power consumption ultra-wide angle monitoring sensor comprises an ultra-wide angle lens,

the NPU is used for starting the low-power consumption ultra-wide angle monitoring sensor and the ultrasonic sensor when the user uses the immersive application software and/or the target condition is within a preset target condition range, the low-power consumption ultra-wide angle monitoring sensor and the ultrasonic sensor are used for monitoring danger, and the target condition comprises time, place or weather;

when the low-power consumption ultra-wide-angle monitoring sensor and/or the ultrasonic sensor monitor a dangerous source, the low-power consumption ultra-wide-angle monitoring sensor and the ultrasonic sensor collect the characteristics of the dangerous source, and the NPU compares the characteristics of the dangerous source with a preset dangerous source characteristic library to obtain a dangerous characteristic value; and when the dangerous characteristic value exceeds a preset characteristic value threshold, the NPU reminds the user.

Images