CN113822449B - Collision detection method, collision detection device, electronic equipment and storage medium - Google Patents
Collision detection method, collision detection device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113822449B CN113822449B CN202110914321.2A CN202110914321A CN113822449B CN 113822449 B CN113822449 B CN 113822449B CN 202110914321 A CN202110914321 A CN 202110914321A CN 113822449 B CN113822449 B CN 113822449B
- Authority
- CN
- China
- Prior art keywords
- collision
- vehicle
- target user
- change data
- acceleration change
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 117
- 230000033001 locomotion Effects 0.000 claims abstract description 75
- 230000001133 acceleration Effects 0.000 claims description 140
- 230000008859 change Effects 0.000 claims description 108
- 238000000034 method Methods 0.000 claims description 17
- 230000002159 abnormal effect Effects 0.000 claims description 13
- 230000006399 behavior Effects 0.000 claims description 8
- 239000004973 liquid crystal related substance Substances 0.000 claims description 7
- 238000005516 engineering process Methods 0.000 abstract description 4
- 238000013473 artificial intelligence Methods 0.000 abstract description 3
- 238000004891 communication Methods 0.000 description 10
- 238000004590 computer program Methods 0.000 description 10
- 238000012545 processing Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 238000004364 calculation method Methods 0.000 description 3
- 230000000739 chaotic effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 206010010071 Coma Diseases 0.000 description 2
- 230000033228 biological regulation Effects 0.000 description 2
- 238000004422 calculation algorithm Methods 0.000 description 2
- 238000005034 decoration Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000010801 machine learning Methods 0.000 description 2
- 238000011897 real-time detection Methods 0.000 description 2
- 206010052770 Coma states Diseases 0.000 description 1
- 208000027418 Wounds and injury Diseases 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 210000000746 body region Anatomy 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 238000012790 confirmation Methods 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 230000006378 damage Effects 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 208000014674 injury Diseases 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/02—Reservations, e.g. for tickets, services or events
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
- G06Q10/00—Administration; Management
- G06Q10/06—Resources, workflows, human or project management; Enterprise or organisation planning; Enterprise or organisation modelling
- G06Q10/063—Operations research, analysis or management
- G06Q10/0631—Resource planning, allocation, distributing or scheduling for enterprises or organisations
- G06Q10/06315—Needs-based resource requirements planning or analysis
-
- G06Q50/40—
Abstract
The disclosure provides a collision detection method, a collision detection device, electronic equipment and a storage medium, relates to the field of artificial intelligence, in particular to an intelligent traffic technology, and can be particularly used in intelligent cities and intelligent traffic scenes. The specific implementation scheme is as follows: acquiring first motion information of a target user side and receiving second motion information of a co-vehicle user side; and detecting a collision event of the vehicle bearing the target user according to the first motion information and the second motion information. The embodiment of the disclosure can improve the accuracy and efficiency of collision detection.
Description
Technical Field
The disclosure relates to the field of artificial intelligence, in particular to an intelligent traffic technology, which can be particularly used in smart cities and intelligent traffic scenes, and particularly relates to a collision detection method, a collision detection device, electronic equipment and a storage medium.
Background
With the development of network technology, the network taxi service provides convenience for the travel of users.
The network taxi is based on an internet technology to construct a service platform, and is connected with a vehicle and a driver meeting the conditions, and the taxi service is provided by integrating supply and demand information.
Disclosure of Invention
The present disclosure provides a collision detection method, apparatus, electronic device, and storage medium.
According to an aspect of the present disclosure, there is provided a collision detection method including:
acquiring first motion information of a target user side and receiving second motion information of a co-vehicle user side;
and detecting a collision event of the vehicle bearing the target user according to the first motion information and the second motion information.
According to another aspect of the present disclosure, there is provided a collision detection apparatus including:
the multi-user motion information acquisition module is used for acquiring first motion information of a target user side and receiving second motion information of a same-car user side;
and the collision event detection module is used for detecting a collision event of a vehicle bearing the target user according to the first motion information and the second motion information.
According to another aspect of the present disclosure, there is provided an electronic device including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the collision detection method of any one of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the collision detection method of any of the embodiments of the present disclosure.
According to another aspect of the present disclosure, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the collision detection method according to any of the embodiments of the present disclosure.
The embodiment of the disclosure can improve the accuracy and efficiency of collision detection.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the disclosure, nor is it intended to be used to limit the scope of the disclosure. Other features of the present disclosure will become apparent from the following specification.
Drawings
The drawings are for a better understanding of the present solution and are not to be construed as limiting the present disclosure. Wherein:
FIG. 1 is a schematic illustration of a collision detection method according to an embodiment of the present disclosure;
FIG. 2 is a schematic illustration of a collision detection method according to an embodiment of the present disclosure;
FIG. 3 is a schematic illustration of a collision detection method according to an embodiment of the present disclosure;
FIG. 4 is a schematic view of a collision detection apparatus according to an embodiment of the present disclosure;
fig. 5 is a block diagram of an electronic device for implementing a collision detection method of an embodiment of the present disclosure.
Detailed Description
Exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding, and should be considered as merely exemplary. Accordingly, one of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted in the following description for clarity and conciseness.
Fig. 1 is a flowchart of a collision detection method according to an embodiment of the present disclosure, which may be applied to a case of acquiring whether a collision event exists in a vehicle driven by a driver or a vehicle on which a passenger sits. The method of the embodiment can be executed by a collision detection device, and the device can be implemented in a software and/or hardware mode and is specifically configured in an electronic device with a certain data operation capability, wherein the electronic device can be a client device, and the client device can be a mobile phone, a tablet computer, a vehicle-mounted terminal, a desktop computer and the like.
S101, acquiring first motion information of a target user side and receiving second motion information of a co-vehicle user side.
The target user side is a user side configured in the electronic device for executing the collision detection method provided by the disclosed embodiment, that is, the target user side is an execution subject. The target user end and the user end for the same vehicle are user ends held by users who ride the same vehicle. For example, in a network-bound vehicle application scenario, a user riding the same vehicle includes a driver or a passenger. The target user side may include a driver side or a passenger side. The on-board user side comprises a driver side and/or a passenger side. The target user end is a driver end, and the same-vehicle user end is a passenger end; or the target user end is a passenger end, and the on-vehicle user end is a driver end and/or a passenger end. In the application scene of carpooling, a plurality of passengers take the same vehicle, and target user end and the same vehicle user end can be passenger end, and the quantity of the same vehicle user end is a plurality of at this moment.
The first movement information may refer to information describing a movement condition of the target user side, and the second movement information may refer to information describing a movement condition of the fellow user side. The number of the same-car user ends is at least one, and the number of the second motion information is at least one correspondingly. The motion information may be real-time motion data or motion information for a period of time. By way of example, the motion information may include at least one of the following speeds, accelerations, angular velocities, and the like. Acceleration can be acquired through an acceleration sensor in an electronic device provided with a user side, angular velocity can be acquired through a gyroscope in the electronic device, velocity can be acquired through a velocity sensor in the electronic device, or velocity can be calculated according to the acquired acceleration, and the like.
The target user side can collect the first motion information of the target user side and directly or indirectly receive the second motion information of the target user side collected by the same-vehicle user side through network communication. The target user end and the on-vehicle user end are both in communication connection with a server, the server is used for providing network vehicle service, and the server receives second motion information uploaded by the on-vehicle user end and forwards the second motion information to the target user end; alternatively, the target client and the on-vehicle client establish a temporary short-range network connection, such as via bluetooth.
S102, detecting a collision event of a vehicle bearing the target user according to the first motion information and the second motion information.
According to the first motion information and the second motion information, under the condition that the included acceleration is larger than or equal to a preset acceleration threshold value, determining that a collision event exists on a vehicle bearing a target user; in the event that it is determined that any of the included accelerations is less than the acceleration threshold, it is determined that there is no collision event with the vehicle carrying the target user. In addition, according to the first motion information and the second motion information, the first acceleration change data and the second acceleration change data can be respectively extracted, and under the condition that the first acceleration change data and the second acceleration change data are matched and the acceleration change conditions of collision are met, a collision event of a vehicle bearing a target user is determined; in the event that the two do not match, or in any case do not meet the acceleration change condition of the collision, it is determined that the vehicle carrying the target user is not in the event of a collision.
The collision event of the vehicle is detected according to the first motion information and the second motion information, and the collision event of the vehicle is actually detected through equipment held by two users who take the same vehicle, so that the motion condition of the vehicle can be accurately described, the collision event of the vehicle bearing the target user is detected, and the detection accuracy of the collision event can be improved.
In the prior art, a vehicle may be mounted with a collision detection apparatus. However, the collision detection device is not installed in each vehicle, and in the case that the target user is on a vehicle without the collision detection device, the collision event of the vehicle cannot be detected in time, so that the user is unsafe to take a bus. In addition, only one piece of motion information is adopted to detect the collision event, and a user may slip down the user end by hand, so that erroneous judgment is caused, and the accuracy of the collision event is low.
According to the technical scheme, the motion information of the user side is used for detection, so that the method can be universally used in the running process of different vehicles, the collision event of the vehicle bearing passengers is effectively detected, the detection instantaneity of the collision event is improved, the real-time detection is effectively processed, the riding safety is improved, the motion condition of the vehicle can be accurately described by detecting the collision event of the vehicle by two devices held by two users sharing one vehicle, the collision event of the vehicle bearing target users is detected, and the detection accuracy of the collision event can be improved.
Fig. 2 is a flowchart of another collision detection method according to an embodiment of the present disclosure, further optimized and expanded based on the above technical solution, and may be combined with the above various alternative embodiments. The detecting a collision event of the vehicle bearing the target user according to the first motion information and the second motion information is embodied as follows: determining first acceleration change data of the target user side according to the first motion information; determining second acceleration change data of the same-vehicle user side according to the second motion information; and detecting a collision event of the vehicle bearing the target user under the condition that the first acceleration change data is matched with the second acceleration change data and the collision condition is met.
S201, obtaining first motion information of a target user side and receiving second motion information of a co-vehicle user side.
S202, determining first acceleration change data of the target user side according to the first motion information.
The first motion information includes acceleration over a period of time. The first acceleration change data may refer to a state in which the magnitude of the acceleration changes over a period of time. For example, acceleration may be periodically collected by an acceleration sensor in an electronic device configured with a target client, and the collection time recorded. According to the acquisition time and the acquired acceleration, first acceleration change data can be acquired. For example, according to the collection time and the collected acceleration, the condition that the value of the acceleration changes along with the collection time is determined, for example, the first acceleration change data is the change condition that the value of the acceleration increases and decreases along with the collection time.
S203, determining second acceleration change data of the same-vehicle user side according to the second motion information.
The second motion information includes acceleration over a period of time. The second acceleration change data may refer to a state in which the magnitude of the acceleration changes over a period of time. The acceleration can be periodically acquired by an acceleration sensor of the electronic equipment configured with the on-vehicle user side, and the acquisition time is recorded. And acquiring second acceleration change data according to the acquisition time and the acquired acceleration.
S204, detecting a collision event of the vehicle bearing the target user according to the first acceleration change data, the second acceleration change data and the collision condition.
And detecting whether a collision event exists in the vehicle of the target user according to the relation between the first acceleration change data and the second acceleration change data, whether the first acceleration change data meets the collision condition or not, and whether the second acceleration change data meets the collision condition or not. For example, it is detected whether the first acceleration change data and the second acceleration change data match, and whether both satisfy a collision condition, and whether a collision event exists in the vehicle of the target user.
Optionally, in the case that the first acceleration change data and the second acceleration change data are matched and both meet a collision condition, determining that a collision event exists for a vehicle carrying the target user.
The first acceleration change data is matched with the second acceleration change data, so that the acceleration change conditions of the target user side and the same-vehicle user side are the same or similar. For example, the similarity between the first acceleration change data and the second acceleration change data may be calculated, and in the case that the similarity is equal to or greater than a preset similarity threshold, it is determined that the first acceleration change data matches the second acceleration change data. The acceleration change data is a curve of acceleration change along with time, a curve model of acceleration change along with time can be obtained by fitting according to each acquisition time and the acceleration corresponding to the acquisition time, the similarity between the two curve models is calculated, and the similarity is determined as the similarity of the first acceleration change data and the second acceleration change data.
The collision condition is used to detect whether the acceleration change data matches the standard acceleration change data at the time of a vehicle collision. In the case where the first acceleration change data matches the second acceleration change data, it may be detected whether only one of the acceleration change data satisfies the collision condition, for example, whether only the first acceleration change data satisfies the collision condition. The collision condition includes standard acceleration change data of the user side at the time of a vehicle collision. Detecting whether the first acceleration change data satisfies the collision condition may be detecting whether a similarity between the first acceleration change data and the standard acceleration change data is equal to or greater than a similarity threshold, for example, determining that the collision condition is satisfied in a case where the similarity is equal to or greater than the similarity threshold; in the case where the similarity is smaller than the similarity threshold, it is determined that the collision condition is not satisfied.
Further, in the case where the first acceleration change data does not match the second acceleration change data, or either of them does not satisfy a collision condition, it is determined that there is no collision event with the vehicle carrying the target user.
By configuring the detection content of the collision event, the method is more in line with the actual collision scene, realizes accurate detection of the collision event, and improves the detection accuracy of the collision event.
Optionally, the first acceleration change data satisfies a collision condition, including: and under the condition that the first acceleration is larger than or equal to a set threshold value in the previous time and is zero in the latter time, determining that the first acceleration change data meets the collision condition.
The first acceleration may include a plurality of accelerations acquired by the target user terminal during a certain period of time. It should be noted that, the direction pointed by the broadside of the user terminal determines the x-axis, the direction pointed by the long side of the user terminal determines the y-axis, and the direction perpendicular to the mobile phone determines the z-axis. The direction of the first acceleration is the same as the direction of impact. The acceleration in each direction may be detected and converted to an acceleration in a coordinate system formed by a plane in which the traveling direction is located after the impact, and determined as the first acceleration.
The time sequence of the previous time precedes the time sequence of the next time, and the previous time and the next time may refer to two time points of the time sequence or two time periods of the time sequence. The previous and subsequent times may be adjacent in time sequence or separated by a period of any duration. The first acceleration is greater than or equal to a set threshold value for determining that the first acceleration is sufficiently large. The set threshold can be obtained according to experimental statistics. The set threshold may be 10m/s 2 。
In fact, the vehicle in which the target user is located encounters an impact, the speed suddenly increases, after the impact, the speed is reduced, eventually immobilized, and correspondingly, the acceleration suddenly increases, eventually becoming zero. The first acceleration is larger than or equal to a set threshold value in the previous time and is zero in the next time, so that the first acceleration is increased rapidly and then reduced to zero, namely the acceleration of the target user terminal is suddenly increased, and finally the target user terminal is stationary.
In the case where the first acceleration is smaller than the set threshold value at the previous time or is non-zero at the latter time, it is determined that the first acceleration change data does not satisfy the collision condition.
By specifically configuring the collision conditions, whether the change trend of the first acceleration accords with the change trend of the acceleration of the user side in a collision vehicle scene or not can be accurately detected, so that whether a collision event exists or not is determined, and the detection accuracy of the collision event is improved.
Optionally, the detecting a collision event of the vehicle carrying the target user when the first acceleration change data and the second acceleration change data are matched and all meet a collision condition includes: under the condition that the first acceleration change data is matched with the second acceleration change data and the collision conditions are met, determining the collision time and acquiring in-vehicle audio data of the collision time; and detecting a collision event of the vehicle bearing the target user according to the in-vehicle audio data.
The audio data may be collected continuously. The collision time may be a time period during which a collision is suspected, and may be determined according to the first acceleration change data, for example, a time range in which there is a surge is queried in the first acceleration change data. And intercepting the audio data of the collision time from the audio data acquired for a long time, and determining the audio data as the audio data in the vehicle of the collision time.
The in-vehicle audio data is used to detect whether a sound of a collision occurs. In fact, in the event of a vehicle collision, a "pop" sound is generated. It may be detected whether the sound is present to further determine whether a collision event has occurred. For example, a section of standard collision sound can be collected in advance, the similarity between the audio data in the vehicle and the standard collision sound is calculated, and the collision event of the vehicle bearing the target user is determined under the condition that the similarity is greater than or equal to the sound similarity; and determining that the collision event does not exist in the vehicle carrying the target user under the condition that the similarity is smaller than the sound similarity. Or, a plurality of segments of standard collision sound can be collected in advance, standard collision sound characteristics are extracted, audio characteristics are extracted from audio data in the vehicle, the similarity between the standard collision sound characteristics and the audio characteristics is calculated, and under the condition that the similarity is greater than or equal to the characteristic similarity, the collision event of the vehicle bearing the target user is determined; and under the condition that the similarity is smaller than the feature similarity, determining that the vehicle carrying the target user has no collision event. The similarity calculation may be implemented by vector calculation or a machine learning algorithm.
By collecting the audio data of the collision time and combining the acceleration change data together to detect the collision event, the collision event can be detected through the sound dimension, the detection dimension of the collision event is increased, and the detection accuracy of the collision event is improved.
According to the technical scheme, the acceleration change data are extracted from the motion information, the relation between the first acceleration change data and the second acceleration change data is extracted, whether the first acceleration change data meets the collision condition or not, whether the second acceleration change data meets the collision condition or not, whether a collision event exists on a vehicle of a target user or not is detected, the motion condition of the vehicle can be accurately described, and whether the acceleration change data meets the collision condition of the acceleration change condition in a real collision scene or not is detected according to the acceleration change data, so that the collision event of the vehicle of the load target user is detected, and the detection accuracy of the collision event can be improved.
Fig. 3 is a flowchart of another collision detection method according to an embodiment of the present disclosure, further optimized and expanded based on the above technical solution, and may be combined with the above various alternative embodiments. The optimization is as follows: executing alarm prompting operation under the condition that the collision event of the vehicle is detected; or under the condition that the collision event of the vehicle is detected, acquiring a state detection result, and under the condition that the state detection result is an abnormal state, executing alarm operation.
S301, acquiring first motion information of a target user side and receiving second motion information of a co-vehicle user side.
S302, detecting a collision event of a vehicle bearing the target user according to the first motion information and the second motion information.
S303, executing alarm prompting operation under the condition that the collision event of the vehicle is detected.
The alarm prompting operation is used for prompting whether the target user holding the target user end alarms or not. The target user can be prompted by text, voice or image, etc. whether to alarm. Information about whether an alarm is given or not can be input by a target user through a key, a specified gesture or voice and the like. For example, the prompt may be "ask if an alarm is needed". If the target user receives the information of the user confirmation alarm, executing alarm operation; otherwise, the target user side does not execute the operation and re-detects whether a collision event occurs. In addition, the target user side can also detect whether the target user inputs a real-time alarm instruction in real time, and the specific input mode can be a mode of pressing a button, designating a gesture, voice or designating an action, wherein the designating action can be to drop the electronic equipment to the ground, so that the target user side executes alarm operation under the condition that the detected acceleration is greater than or equal to a preset alarm threshold value.
Wherein, performing the alarm operation may include: sending alarm information to an official safety system, wherein the alarm information comprises information of a driving user, information of a riding user, latest positioning information and the like; sending alarm information to a designated emergency contact or establishing call connection; sending alarm information to business personnel of the taxi taking service system or establishing call connection, etc.
S304, or in the case that the collision event of the vehicle is detected, a state detection result is obtained, and in the case that the state detection result is an abnormal state, an alarm operation is executed.
The state detection result is used to detect whether a state in a serious car accident occurs, and the state may include a vehicle state, an in-car decorator state, a person state, and the like. The abnormal state of the state detection result indicates that a collision event of a serious car accident occurs, and the target user cannot alarm in time or cannot alarm, for example, the target user is in coma. In addition, in the case where the state detection result is a normal state, an alarm prompting operation is performed.
Optionally, the obtaining the status detection result includes: acquiring positioning information of the target user side; and detecting yaw behavior according to the positioning information of the target user side and the designated navigation route, and determining a state detection result.
The positioning information of the target user side is used for determining a yaw designated navigation route of the target user side. Under normal conditions, the target user side designates the driving route of the network about vehicle, namely, the driving route is the designated navigation route, and the driving user drives the vehicle along the designated navigation route. And under the condition that the positioning information of the target user side does not belong to the point set of the designated navigation route, determining that the target user side deviates from the designated navigation route, detecting that the vehicle has yaw behavior, indicating that the vehicle is collided and deviates from the original driving route, and thus indicating that the vehicle is severely collided. Under the condition that yaw behavior is detected, determining a state detection result as an abnormal state; in the case where the presence of yaw behavior is not detected, it is determined that the state detection result is a normal state.
The yaw behavior is detected through the positioning information, so that the serious collision condition can be detected rapidly, and the state detection result is determined, so that whether the alarm prompt operation is skipped or not is determined, the alarm operation is directly executed, the processing efficiency of collision events is improved, and the riding safety is improved.
Optionally, the obtaining the status detection result includes: image acquisition is carried out on the interior of the vehicle; detecting an object of the acquired image and determining the state of the object; and determining a state detection result according to the object state.
The camera of the electronic equipment can be called by the target user side to collect images of the interior of the vehicle in the shooting range. Object detection for identifying a living body and a decoration in the acquired image and determining a position of the living body in the image and a position of the decoration in the image. The object state of the living body may refer to whether the living body is in an abnormal state, wherein the abnormal state includes coma or injury. The object state of the ornament may refer to whether the ornament is in a chaotic state. Determining that the state detection result is an abnormal state when the object state of the living body is an abnormal state and/or the object state of the ornament is a chaotic state; when the object state of the living body is a normal state and the object state of the ornament is an ordered state, the state detection result is determined to be the normal state, and the object state of the living body being the normal state means that the living body is awake and not injured. The state detection result is an abnormal state, which indicates that the vehicle is in serious collision; the state detection result is a normal state, which indicates that the vehicle is not in serious collision.
Wherein the number of images is at least one. An image can be acquired, whether blood stains exist in the image is detected, and if so, the object state of the living body is determined to be an injured state; otherwise, the object state is determined to be an uninjured state. A plurality of images may be continuously acquired, and the object state of the living body may be detected from the living body region in each image. For example, whether the posture of the living body in the living body area in the different images is changed is detected, and if the living body is determined to be unchanged, the object state of the living body is determined to be a coma state; in the case where the change is determined to occur, the object state of the living body is determined to be an awake state. For example, whether the eyes of the living body are in a closed state within a preset time period can be detected, and if so, no change is determined; otherwise, determining that the change occurs; or detecting whether the living body is in a lying posture within a preset time period, and if so, determining that the living body is unchanged; otherwise, the change is determined to occur. The preset time period may be set as required, for example, 5 seconds. When the object state of the living body is an uninjured state and an awake state, the state of the living body is determined to be a normal state.
An image may be acquired and the object state of the ornament detected based on a plurality of ornament areas in the image. Exemplary, detect the direction of putting of ornaments in different dress areas in the picture is the standard direction, if yes, confirm the object state of the ornaments as the ordered state; otherwise, determining that the object state of the ornament is a chaotic state. The standard direction refers to the direction in which the ornament is placed in the normal state, and the placement direction is the standard direction, so that the ornament is placed in the normal state. The photo frame is characterized in that when the photo frame is in a normal placement direction, the short side is vertical to the ground, and the long side is parallel to the ground, so that whether the angle between the short side of the photo frame and the ground is 90 degrees can be detected, and the placement direction is determined to be a standard direction under the condition that the angle is 90 degrees; otherwise, the direction is not the standard direction.
Through image acquisition, state detection is carried out on the object identified in the image, the object state is determined according to the object state, and the serious collision condition can be rapidly detected, so that the state detection result is determined, whether the alarm prompt operation is skipped or not is determined, the alarm operation is directly executed, the processing efficiency of collision events is improved, and the riding safety is improved.
According to the technical scheme, the alarm prompt operation is executed under the condition that the collision event exists, and the alarm operation is executed when the abnormal state is detected, so that the collision event can be timely alarm-processed, the real-time processing performance of the collision event is improved, and the riding safety is improved.
Fig. 4 is a block diagram of a collision detection apparatus in an embodiment of the present disclosure, which is applicable to a case of acquiring whether a collision event exists in a vehicle driven by a driver or a vehicle on which a passenger sits, according to an embodiment of the present disclosure. The device is realized by software and/or hardware, and is specifically configured in an electronic device with certain data operation capability, wherein the electronic device can be a target user terminal.
A collision detection apparatus 400 as shown in fig. 4, comprising: a multi-user motion information acquisition module 401 and a collision event detection module 402; wherein, the liquid crystal display device comprises a liquid crystal display device,
the multi-user motion information acquisition module 401 is configured to acquire first motion information of a target user terminal and receive second motion information of a co-vehicle user terminal;
a collision event detection module 402, configured to detect a collision event of a vehicle carrying the target user according to the first motion information and the second motion information.
According to the technical scheme, the motion information of the user side is used for detection, so that the method can be universally used in the running process of different vehicles, the collision event of the vehicle bearing passengers is effectively detected, the detection instantaneity of the collision event is improved, the real-time detection is effectively processed, the riding safety is improved, the motion condition of the vehicle can be accurately described by detecting the collision event of the vehicle by two devices held by two users sharing one vehicle, the collision event of the vehicle bearing target users is detected, and the detection accuracy of the collision event can be improved.
Further, the collision event detection module 402 includes: the first acceleration change data detection unit is used for determining first acceleration change data of the target user side according to the first motion information; the second acceleration change data detection unit is used for determining second acceleration change data of the on-vehicle user side according to the second motion information; and the collision event detection unit is used for detecting a collision event of the vehicle bearing the target user according to the first acceleration change data, the second acceleration change data and the collision condition.
Further, the collision event detection unit includes: and the collision condition detection subunit is used for determining that a collision event exists in the vehicle bearing the target user under the condition that the first acceleration change data is matched with the second acceleration change data and the collision condition is met.
Further, the collision event detection unit includes: a collision audio acquiring subunit, configured to determine a collision time and acquire in-vehicle audio data of the collision time when the first acceleration change data and the second acceleration change data are matched and all meet a collision condition; and the collision detection subunit is used for detecting a collision event of the vehicle bearing the target user according to the in-vehicle audio data.
Further, the collision detection device further includes: the alarm prompting module is used for executing alarm prompting operation under the condition that the collision event of the vehicle is detected; or the alarm execution module is used for acquiring a state detection result under the condition that the collision event of the vehicle is detected, and executing alarm operation under the condition that the state detection result is an abnormal state.
Further, the alarm execution module includes: the positioning information acquisition unit is used for acquiring the positioning information of the target user side; and the state detection result determining unit is used for detecting yaw behaviors according to the positioning information of the target user side and the designated navigation route and determining a state detection result.
Further, the alarm execution module includes: the image acquisition unit is used for acquiring images of the interior of the vehicle; the object state detection unit is used for detecting an object of the acquired image and determining the state of the object; and the state detection result determining unit is used for determining a state detection result according to the object state.
The collision detection device can execute the collision detection method provided by any embodiment of the disclosure, and has the corresponding functional modules and beneficial effects of executing the collision detection method.
In the technical scheme of the disclosure, the acquisition, storage, application and the like of the related user personal information or vehicle information all conform to the regulations of related laws and regulations, and the public order is not violated.
According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
Fig. 5 illustrates a schematic block diagram of an example electronic device 500 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the disclosure described and/or claimed herein.
As shown in fig. 5, the apparatus 500 includes a computing unit 501 that can perform various suitable actions and processes according to a computer program stored in a Read Only Memory (ROM) 502 or a computer program loaded from a storage unit 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, ROM 502, and RAM 503 are connected to each other by a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
Various components in the device 500 are connected to the I/O interface 505, including: an input unit 506 such as a keyboard, a mouse, etc.; an output unit 507 such as various types of displays, speakers, and the like; a storage unit 508 such as a magnetic disk, an optical disk, or the like; and a communication unit 509 such as a network card, modem, wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information/data with other devices via a computer network such as the internet and/or various telecommunication networks.
The computing unit 501 may be a variety of general and/or special purpose processing components having processing and computing capabilities. Some examples of computing unit 501 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various computing units running machine learning model algorithms, a Digital Signal Processor (DSP), and any suitable processor, controller, microcontroller, etc. The calculation unit 501 performs the respective methods and processes described above, such as a collision detection method. For example, in some embodiments, the collision detection method may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 508. In some embodiments, part or all of the computer program may be loaded and/or installed onto the device 500 via the ROM 502 and/or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the collision detection method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the collision detection method by any other suitable means (e.g. by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
Program code for carrying out methods of the present disclosure may be written in any combination of one or more programming languages. These program code may be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus such that the program code, when executed by the processor or controller, causes the functions/operations specified in the flowchart and/or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and pointing device (e.g., a mouse or trackball) by which a user can provide input to the computer. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), and the internet.
The computer system may include a client and a server. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server, a server of a distributed system, or a server incorporating a blockchain.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure may be performed in parallel, sequentially, or in a different order, provided that the desired results of the technical solutions provided by the present disclosure are achieved, and are not limited herein.
The above detailed description should not be taken as limiting the scope of the present disclosure. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present disclosure are intended to be included within the scope of the present disclosure.
Claims (12)
1. A collision detection method, comprising:
acquiring first motion information of a target user side and receiving second motion information of a co-vehicle user side; the target user end is a driver end, and the same-vehicle user end is a passenger end; or the target user end is a passenger end, and the co-vehicle user end is a driver end and/or a passenger end;
determining first acceleration change data of the target user side according to the first motion information;
Determining second acceleration change data of the same-vehicle user side according to the second motion information;
detecting a collision event of a vehicle carrying the target user according to the first acceleration change data, the second acceleration change data and collision conditions;
wherein the detecting a collision event of the vehicle carrying the target user according to the first acceleration change data, the second acceleration change data and the collision condition comprises:
and under the condition that the first acceleration change data is matched with the second acceleration change data and the collision condition is met, determining that a collision event exists in a vehicle bearing the target user.
2. The method of claim 1, wherein the detecting a collision event of a vehicle carrying the target user if the first acceleration change data matches the second acceleration change data and both satisfy a collision condition comprises:
under the condition that the first acceleration change data is matched with the second acceleration change data and the collision conditions are met, determining the collision time and acquiring in-vehicle audio data of the collision time;
And detecting a collision event of the vehicle bearing the target user according to the in-vehicle audio data.
3. The method of claim 1, further comprising:
executing alarm prompting operation under the condition that the collision event of the vehicle is detected; or alternatively
And acquiring a state detection result when the collision event of the vehicle is detected, and executing alarm operation when the state detection result is an abnormal state.
4. A method according to claim 3, wherein the obtaining a status detection result comprises:
acquiring positioning information of the target user side;
and detecting yaw behavior according to the positioning information of the target user side and the designated navigation route, and determining a state detection result.
5. A method according to claim 3, wherein the obtaining a status detection result comprises:
image acquisition is carried out on the interior of the vehicle;
detecting an object of the acquired image and determining the state of the object;
and determining a state detection result according to the object state.
6. A collision detection apparatus comprising:
the multi-user motion information acquisition module is used for acquiring first motion information of a target user side and receiving second motion information of a same-car user side; the target user end is a driver end, and the same-vehicle user end is a passenger end; or the target user end is a passenger end, and the co-vehicle user end is a driver end and/or a passenger end;
A collision event detection module, comprising:
the first acceleration change data detection unit is used for determining first acceleration change data of the target user side according to the first motion information;
the second acceleration change data detection unit is used for determining second acceleration change data of the on-vehicle user side according to the second motion information;
a collision event detection unit configured to detect a collision event of a vehicle carrying the target user based on the first acceleration change data, the second acceleration change data, and a collision condition;
wherein the collision event detection unit includes:
and the collision condition detection subunit is used for determining that a collision event exists in the vehicle bearing the target user under the condition that the first acceleration change data is matched with the second acceleration change data and the collision condition is met.
7. The apparatus of claim 6, wherein the collision event detection unit comprises:
a collision audio acquiring subunit, configured to determine a collision time and acquire in-vehicle audio data of the collision time when the first acceleration change data and the second acceleration change data are matched and all meet a collision condition;
And the collision detection subunit is used for detecting a collision event of the vehicle bearing the target user according to the in-vehicle audio data.
8. The apparatus of claim 6, further comprising:
the alarm prompting module is used for executing alarm prompting operation under the condition that the collision event of the vehicle is detected; or alternatively
And the alarm execution module is used for acquiring a state detection result under the condition that the collision event of the vehicle is detected, and executing alarm operation under the condition that the state detection result is an abnormal state.
9. The apparatus of claim 8, wherein the alert execution module comprises:
the positioning information acquisition unit is used for acquiring the positioning information of the target user side;
and the state detection result determining unit is used for detecting yaw behaviors according to the positioning information of the target user side and the designated navigation route and determining a state detection result.
10. The apparatus of claim 8, wherein the alert execution module comprises:
the image acquisition unit is used for acquiring images of the interior of the vehicle;
the object state detection unit is used for detecting an object of the acquired image and determining the state of the object;
And the state detection result determining unit is used for determining a state detection result according to the object state.
11. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the collision detection method of any one of claims 1-5.
12. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the collision detection method according to any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914321.2A CN113822449B (en) | 2021-08-10 | 2021-08-10 | Collision detection method, collision detection device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110914321.2A CN113822449B (en) | 2021-08-10 | 2021-08-10 | Collision detection method, collision detection device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113822449A CN113822449A (en) | 2021-12-21 |
| CN113822449B true CN113822449B (en) | 2023-08-18 |
Family
ID=78922792
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110914321.2A Active CN113822449B (en) | 2021-08-10 | 2021-08-10 | Collision detection method, collision detection device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113822449B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1989029A (en) * | 2004-07-14 | 2007-06-27 | Trw汽车美国有限责任公司 | Method and apparatus for controlling an actuatable restraining device using crush zone sensors for safing function |
| WO2013105917A2 (en) * | 2011-12-02 | 2013-07-18 | Intel Corporation | Automatic accident analysis |
| US9650007B1 (en) * | 2015-04-13 | 2017-05-16 | Allstate Insurance Company | Automatic crash detection |
| CN107463907A (en) * | 2017-08-08 | 2017-12-12 | 东软集团股份有限公司 | Vehicle collision detection method, device, electronic equipment and vehicle |
| US9934625B1 (en) * | 2017-01-31 | 2018-04-03 | Uber Technologies, Inc. | Detecting vehicle collisions based on moble computing device data |
| CN108422959A (en) * | 2017-02-13 | 2018-08-21 | 丰田自动车株式会社 | Vehicle collision detecting system and recording medium |
| CN109564734A (en) * | 2016-08-22 | 2019-04-02 | 索尼公司 | Drive assistance device, driving assistance method, moving body and program |
| CN110426215A (en) * | 2019-07-16 | 2019-11-08 | 驭势（上海）汽车科技有限公司 | A kind of method for establishing model and intelligent driving system for vehicle ride comfort test |
| CN110481534A (en) * | 2018-05-10 | 2019-11-22 | 本田技研工业株式会社 | Vehicle control system, control method for vehicle and storage medium |
| CN111665011A (en) * | 2019-03-07 | 2020-09-15 | 北京奇虎科技有限公司 | Collision detection method and device |
| CN112153566A (en) * | 2020-11-26 | 2020-12-29 | 博泰车联网(南京)有限公司 | Method, computing device and computer storage medium for customer service |
| CN112277936A (en) * | 2020-10-10 | 2021-01-29 | 广州亚美智造科技有限公司 | Vehicle collision detection processing method and device, vehicle-mounted terminal and storage medium |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9818239B2 (en) * | 2015-08-20 | 2017-11-14 | Zendrive, Inc. | Method for smartphone-based accident detection |
-
2021
- 2021-08-10 CN CN202110914321.2A patent/CN113822449B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1989029A (en) * | 2004-07-14 | 2007-06-27 | Trw汽车美国有限责任公司 | Method and apparatus for controlling an actuatable restraining device using crush zone sensors for safing function |
| WO2013105917A2 (en) * | 2011-12-02 | 2013-07-18 | Intel Corporation | Automatic accident analysis |
| US9650007B1 (en) * | 2015-04-13 | 2017-05-16 | Allstate Insurance Company | Automatic crash detection |
| CN109564734A (en) * | 2016-08-22 | 2019-04-02 | 索尼公司 | Drive assistance device, driving assistance method, moving body and program |
| US9934625B1 (en) * | 2017-01-31 | 2018-04-03 | Uber Technologies, Inc. | Detecting vehicle collisions based on moble computing device data |
| CN108422959A (en) * | 2017-02-13 | 2018-08-21 | 丰田自动车株式会社 | Vehicle collision detecting system and recording medium |
| CN107463907A (en) * | 2017-08-08 | 2017-12-12 | 东软集团股份有限公司 | Vehicle collision detection method, device, electronic equipment and vehicle |
| CN110481534A (en) * | 2018-05-10 | 2019-11-22 | 本田技研工业株式会社 | Vehicle control system, control method for vehicle and storage medium |
| CN111665011A (en) * | 2019-03-07 | 2020-09-15 | 北京奇虎科技有限公司 | Collision detection method and device |
| CN110426215A (en) * | 2019-07-16 | 2019-11-08 | 驭势（上海）汽车科技有限公司 | A kind of method for establishing model and intelligent driving system for vehicle ride comfort test |
| CN112277936A (en) * | 2020-10-10 | 2021-01-29 | 广州亚美智造科技有限公司 | Vehicle collision detection processing method and device, vehicle-mounted terminal and storage medium |
| CN112153566A (en) * | 2020-11-26 | 2020-12-29 | 博泰车联网(南京)有限公司 | Method, computing device and computer storage medium for customer service |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113822449A (en) | 2021-12-21 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7399075B2 (en) | Information processing device, information processing method and program | |
| US10812952B2 (en) | Systems and methods for detecting driver phone operation using vehicle dynamics data | |
| KR102205240B1 (en) | Unexpected Impulse Change Collision Detector | |
| CN112513787B (en) | Interaction method, electronic device and system for in-vehicle isolation gesture | |
| CN105023394A (en) | Dangerous driving reminding and controlling method based on portable intelligent device | |
| CN105139583A (en) | Vehicle danger prompting method based on portable intelligent equipment | |
| CN112269865A (en) | Method, device, equipment and storage medium for generating information | |
| CN113569718A (en) | Reminding method, device, equipment, system and storage medium | |
| US9080880B2 (en) | Motion based message delivery | |
| CN112689587A (en) | Method for classifying non-driving task activities in consideration of interruptability of non-driving task activities of driver when taking over driving task is required and method for releasing non-driving task activities again after non-driving task activities are interrupted due to taking over driving task is required | |
| JP7186749B2 (en) | Management system, management method, management device, program and communication terminal | |
| CN113822449B (en) | Collision detection method, collision detection device, electronic equipment and storage medium | |
| CN114677848B (en) | Perception early warning system, method, device and computer program product | |
| US20200149907A1 (en) | Vehicular apparatus, systems, and methods for detecting, identifying, imaging, and mapping noise sources | |
| CN114333269A (en) | Vehicle accident handling method and device, computer equipment and storage medium | |
| CN114852068A (en) | Pedestrian collision avoidance method, device, equipment and storage medium | |
| CN113494921B (en) | Automatic driving method and device | |
| CN113147794A (en) | Method, device and equipment for generating automatic driving early warning information and automatic driving vehicle | |
| CN115320626B (en) | Danger perception capability prediction method and device based on human-vehicle state and electronic equipment | |
| CN107222627B (en) | Automobile driver identification method based on smart phone | |
| EP4276788A1 (en) | Improvement item detection device, improvement item detection method, and program | |
| CN117915041A (en) | Control method, device, equipment and medium for vehicle-mounted panoramic image display picture | |
| CN114379585A (en) | Method, apparatus, device and storage medium for displaying vehicle information | |
| CN116453374A (en) | Information display method and device, electronic equipment and storage medium | |
| CN115830871A (en) | Method and system for monitoring driving safety of in-site motor vehicle and monitoring cloud platform |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |