CN109815805A - Automatic identification drowned method, apparatus, storage medium and electronic equipment - Google Patents
Automatic identification drowned method, apparatus, storage medium and electronic equipment Download PDFInfo
- Publication number
- CN109815805A CN109815805A CN201811554491.9A CN201811554491A CN109815805A CN 109815805 A CN109815805 A CN 109815805A CN 201811554491 A CN201811554491 A CN 201811554491A CN 109815805 A CN109815805 A CN 109815805A
- Authority
- CN
- China
- Prior art keywords
- static frames
- personage
- video
- learning model
- static
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 58
- 230000003068 static effect Effects 0.000 claims abstract description 144
- 230000009182 swimming Effects 0.000 claims abstract description 55
- 238000010801 machine learning Methods 0.000 claims abstract description 48
- 238000012549 training Methods 0.000 claims abstract description 9
- 238000012544 monitoring process Methods 0.000 claims description 18
- 239000000284 extract Substances 0.000 claims description 7
- 238000012163 sequencing technique Methods 0.000 claims description 7
- 238000000605 extraction Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 4
- 238000012545 processing Methods 0.000 description 16
- 206010013647 Drowning Diseases 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 238000004891 communication Methods 0.000 description 5
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 230000005291 magnetic effect Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000013507 mapping Methods 0.000 description 2
- 239000003643 water by type Substances 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 210000004027 cell Anatomy 0.000 description 1
- 239000000470 constituent Substances 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 235000013399 edible fruits Nutrition 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000009434 installation Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003014 reinforcing effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 210000000352 storage cell Anatomy 0.000 description 1
Landscapes
- Image Analysis (AREA)
Abstract
Present disclose provides method and device, storage medium and electronic equipments that a kind of automatic identification is drowned, this method comprises: receiving the video for the image capture device acquisition arranged around swimming pool；Static frames are extracted at predetermined time intervals from the video of the acquisition；Personage is identified from the static frames；If identifying personage from the static frames, determines and identify that the static frames of personage are target frame, the personage recognized is target person；From the static frames continuously extracted after the target frame, the static frames for identifying the predetermined number of the target person are determined；The static frames of the predetermined number are inputted into trained machine learning model, obtain the judging result whether target person drowns.The disclosure realizes automatic, accurate and efficient drowned identification, rescue efficiency when occurring drowned has been effectively ensured by training machine learning model.
Description
Technical field
This disclosure relates to machine learning applied technical field, and in particular to a kind of drowned method, apparatus of automatic identification is deposited
Storage media and electronic equipment.
Background technique
At present although domestic swimming pool is equipped with monitoring and the first-aid personnel of swimming pool, but often due to artificial keeps an eye on not
It in place or absent-mindedness, still can some drowning events frequent occurrence.These tragedies may occur at some without security protection
River or lake in, child leads to drowning incident alone in playing in river etc. due to unattended.
In existing scheme and counter-measure, due to the not in place or carelessness artificially kept an eye on, some drown often occurs
Water death incident；Alternatively, rescue can all bring serious consequence not in time.
Therefore, whether a kind of swimming personnel that can monitor each waters automatically, automatically, efficiently and accurately judge personnel
Drowned method and apparatus are significant, and rescue efficiency can be effectively ensured.
It is therefore desirable to provide a kind of method, apparatus, storage medium and electronic equipment that new automatic identification is drowned.
It should be noted that information is only used for reinforcing the reason to the background of the disclosure disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
The disclosure is designed to provide a kind of method, apparatus, storage medium and electronic equipment that automatic identification is drowned, can
To monitor the swimming personnel in each waters automatically, the method and apparatus meaning whether personnel drown automatically, is efficiently and accurately judged
Justice is great, and rescue efficiency can be effectively ensured.
According to one aspect of the disclosure, a kind of method that automatic identification is drowned is provided, comprising:
Receive the video for the image capture device acquisition arranged around swimming pool；
Static frames are extracted at predetermined time intervals from the video of the acquisition；
Personage is identified from the static frames；
If identifying personage from the static frames, determines and identify that the static frames of personage are target frame, the knowledge
The personage being clipped to is target person；
From the static frames continuously extracted after the target frame, the predetermined number for identifying the target person is determined
Static frames；
The static frames of the predetermined number are inputted into trained machine learning model, obtain whether the target person drowns
The judging result of water.
In a kind of exemplary embodiment of the disclosure, the image capture device acquisition arranged around swimming pool is being received
Before video, the method also includes:
The human face photo for receiving the shooting of swimming pool inlet face photographing device, by the body of the human face photo and the person of being taken
Part information association storage.
It is described to identify personage from the static frames in a kind of exemplary embodiment of the disclosure, comprising:
Face is identified from the static frames using recognition of face；
The face identified is compared with the human face photo of associated storage, comparison result is obtained, to identify people
Object；
The face such as identified is consistent with the human face photo of associated storage, then using the identity information of associated storage as identification
Identity of personage information out.
In a kind of exemplary embodiment of the disclosure, the static frames by the predetermined number input trained machine
Device learning model, comprising:
By the static frame sequence of the predetermined number according to the sequencing of each static frames at the time of point in video, according to
Secondary series connection obtains static frame sequence；
By the trained machine learning model of static frames sequence inputting.
In a kind of exemplary embodiment of the disclosure, the training method of the machine learning model is:
Calibration in advance is collected drowned positive sample occurs and the static frame sequence of drowned negative sample does not occur as sample
Collection；
Each static frames sequence samples in sample set are inputted into machine learning model；
If the judging result of machine learning model output and inconsistent to the calibration of sample in advance, adjustment machine learning model
Coefficient, make to control machine learning model when input is positive sample, drowned judging result occurs for output, is negative sample in input
Drowned judging result does not occur for this when, output；
If machine learning model output judging result with it is consistent to the calibration of sample in advance, train terminate, trained
Good machine learning model.
In a kind of exemplary embodiment of the disclosure, it includes being arranged around swimming pool that described image, which acquires equipment, is used for
Monitor the multiple images acquisition equipment in different swimming regions, which is characterized in that if described identify personage from the static frames,
It determines and identifies that the static frames of personage are target frame, the personage recognized is target person, comprising:
The target frame includes multiple target frames from the video obtained in different image capture devices；
The method also includes:
The multiple target frame at the time of point in the video in the target frame source is obtained respectively；
Obtain the location information of the image capture device of the video in the multiple target frame source；
According to the sequencing of the moment point, it is sequentially connected the position of described image acquisition equipment on the electronic map,
Obtain the motion profile of target person.
In a kind of exemplary embodiment of the disclosure, it includes being arranged around swimming pool that described image, which acquires equipment, is used for
Monitor the multiple images acquisition equipment in different swimming regions, which is characterized in that the static frames by the predetermined number input
Trained machine learning model, after obtaining the judging result whether target person drowns, the method also includes:
If the static frames input of the predetermined number in video in one of image capture device is trained
Machine learning model, obtaining the target person is drowned judging result, obtains the static frames source of the predetermined number
Image capture device position as target position；
Obtain the lifeguard position that the electronic equipment that all lifeguards wear reports；
Calculate each lifeguard position to the target position distance；
Assistance requests are issued to the electronic equipment worn apart from the smallest lifeguard, are carried in the assistance requests
The target position.
According to one aspect of the disclosure, a kind of device that automatic identification is drowned is provided characterized by comprising
Video acquiring module receives the video for the image capture device acquisition arranged around swimming pool；
Extraction module extracts static frames from the video of the acquisition at predetermined time intervals；
Identification module identifies personage from the static frames；
First determining module determines that the static frames for identifying personage are if identifying personage from the static frames
Target frame, the personage recognized are target person；
Second determining module, from the static frames continuously extracted after the target frame, determination identifies the target
The static frames of the predetermined number of personage；
The static frames of the predetermined number are inputted trained machine learning model, obtain the target by judgment module
The judging result whether personage drowns.
According to one aspect of the disclosure, a kind of computer readable storage medium is provided, computer program is stored thereon with,
It is characterized in that, realizing claim 1-7 described in any item methods when the computer program is executed by processor.
According to one aspect of the disclosure, a kind of electronic equipment is provided characterized by comprising
Memory executes computer-readable instruction for storing；
Processor is required any in 1-7 for executing the computer-readable instruction stored in memory with perform claim
A method.
A kind of automatic identification of the disclosure drowned method, apparatus, storage medium and electronic equipment, firstly, receiving in swimming pool
The video for the image capture device acquisition that surrounding is arranged；It, can be with by arranging at least one image capture device around swimming pool
The comprehensive monitoring for realizing swimming pool guarantees the effective image acquisition of each region.Then, from the video of the acquisition every
Predetermined time extracts static frames；Static frames are extracted from collected video can be used to identify personage in subsequent step, every
Calculation processing load can be effectively reduced under the premise of guaranteed efficiency in predetermined time extraction.Then, know from the static frames
Others' object；Face can be accurately identified from video frame by recognition of face, and then people is determined by database matching
Object.If identifying personage from the static frames, determine that the static frames are target frame, the personage recognized is target person
Object；After determining personage, this determine the video frame of personage be exactly can with personage described in effecting reaction whether An Quan target frame,
And then the judgement that whether can be drowned in subsequent for target person, realize the safety of the clear information of each personage in swimming pool
Monitoring.Then, from the static frames continuously extracted after the target frame, the predetermined number for identifying the target person is determined
Purpose static frames；Pass through obtained after determining target frame include target person multiple frames, so that it may using it is drowned when with
Whether the variation accurate judgement personage of video frame drowns when normal swimming, and the accuracy rate of judgement is effectively ensured.Finally, by institute
The static frames for stating predetermined number input trained machine learning model, obtain the judgement the knot whether target person drowns
Fruit；Pass through training machine learning model, so that it may realize it is automatic, efficiently and accurately judge whether personnel drown, and then can be with
Rescue efficiency is effectively ensured.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.
Fig. 1 schematically shows a kind of flow chart of method that automatic identification is drowned.
Fig. 2 schematically shows a kind of Application Scenarios-Example figure of method that automatic identification is drowned.
Fig. 3 schematically shows a kind of method flow diagram that personage is identified from the static frames.
Fig. 4 schematically shows a kind of block diagram of device that automatic identification is drowned.
Fig. 5 schematically shows a kind of electronic equipment example block diagram of method drowned for realizing above-mentioned automatic identification.
Fig. 6 schematically shows a kind of computer readable storage medium of method drowned for realizing above-mentioned automatic identification.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described feature, knot
Structure or characteristic can be incorporated in any suitable manner in one or more embodiments.In the following description, it provides perhaps
More details fully understand embodiment of the present disclosure to provide.It will be appreciated, however, by one skilled in the art that can
It is omitted with technical solution of the disclosure one or more in the specific detail, or others side can be used
Method, constituent element, device, step etc..In other cases, be not shown in detail or describe known solution to avoid a presumptuous guest usurps the role of the host and
So that all aspects of this disclosure thicken.
In addition, attached drawing is only the schematic illustrations of the disclosure, it is not necessarily drawn to scale.Identical attached drawing mark in figure
Note indicates same or similar part, thus will omit repetition thereof.Some block diagrams shown in the drawings are function
Energy entity, not necessarily must be corresponding with physically or logically independent entity.These function can be realized using software form
Energy entity, or these functional entitys are realized in one or more hardware modules or integrated circuit, or at heterogeneous networks and/or place
These functional entitys are realized in reason device device and/or microcontroller device.
Provide firstly the drowned method of automatic identification in this example embodiment, the drowned method of the automatic identification can be with
Server is run on, server cluster or Cloud Server etc. can also be run on, certainly, those skilled in the art can also basis
Demand runs method of the invention in other platforms, and particular determination is not done to this in the present exemplary embodiment.Refering to what is shown in Fig. 1,
The drowned method of the automatic identification may comprise steps of:
Step S110. receives the video for the image capture device acquisition arranged around swimming pool.
Step S120. extracts static frames from the video of the acquisition at predetermined time intervals.
Step S130. identifies personage from the static frames.
If step S140. identifies personage from the static frames, determines and identify that the static frames of personage are target
Frame, the personage recognized are target person.
For step S150. from the static frames continuously extracted after the target frame, determination identifies the target person
Predetermined number static frames.
The static frames of the predetermined number are inputted trained machine learning model by step S160., obtain the target
The judging result whether personage drowns.
In the drowned method of above-mentioned automatic identification, firstly, receiving the image capture device acquisition arranged around swimming pool
Video；By arranging at least one image capture device around swimming pool, the comprehensive monitoring of swimming pool may be implemented, guarantee each
The effective image in a region acquires.Then, static frames are extracted at predetermined time intervals from the video of the acquisition；From collected
Static frames are extracted in video can be used to identify personage in subsequent step, and extracting every the predetermined time can be before guaranteed efficiency
It puts, calculation processing load is effectively reduced.Then, personage is identified from the static frames；It can be accurate by recognition of face
Face is identified from video frame, and then personage is determined by database matching.If identifying personage from the static frames,
Determine that the static frames are target frame, the personage recognized is target person；After determining personage, this determines personage's
Video frame be exactly can with personage described in effecting reaction whether An Quan target frame, and then can be for target person in subsequent
The security monitoring of the clear information of each personage in swimming pool is realized in no drowned judgement.Then, continuous after the target frame
In the static frames extracted, the static frames for identifying the predetermined number of the target person are determined；By in determining target frame
Multiple frames comprising target person are obtained later, so that it may utilize variation accurate judgement when drowning with video frame when normal swim
Whether personage drowns, and the accuracy rate of judgement is effectively ensured.Finally, the static frames input by the predetermined number is trained
Machine learning model obtains the judging result whether target person drowns；Pass through training machine learning model, so that it may real
Now judge whether personnel drown automatically, efficiently and accurately, and then rescue efficiency can be effectively ensured.
In the following, will be carried out in conjunction with attached drawing to each step in automatic information input method above-mentioned in this example embodiment detailed
Thin explanation and explanation.
Receives the video for the image capture device acquisition arranged around swimming pool in step s 110.
In this exemplary embodiment, refering to what is shown in Fig. 2, server 201 receives the Image Acquisition arranged around swimming pool
After the video that equipment 202 acquires, it is analyzed and processed in the next steps；Wherein the server can be at least one and have
Calculating, storage, the equipment, such as computer, microprocessor of processing capacity etc., do not do particular determination, image capture device can herein
To be any equipment with image collecting function, such as mobile phone, camera etc., particular determination is not done herein.
It here may include arranging upper some Image Acquisition in these places on pool edge and under swimming-pool water around swimming pool
Equipment realizes the conduct monitoring at all levels of swimming pool, and then guarantees the effective image acquisition of each region, the connection of these image capture devices
To server, realizes real-time monitoring processing, treatment effeciency is effectively ensured.In one embodiment, these image capture devices
It is evenly arranged in around swimming pool, it is ensured that there is image capture device monitoring in every nook and cranny；For example, can be by the swimming of square
Pond is divided into 4 × 4 square, forms 16 regions.In each region or on pool edge or at the bottom, one is placed
A image capture device is formed and is monitored without dead angle, the video of server real-time reception image capture device acquisition.Implement at one
In example, described image acquisition equipment, which is arranged in, there is a possibility that drowned region, for example, the depth of water is more than 0.5 meter of position.
In a kind of originally exemplary embodiment, in the video for receiving the image capture device acquisition arranged around swimming pool
Before, the method also includes:
The human face photo for receiving the shooting of swimming pool inlet face photographing device, by the body of the human face photo and the person of being taken
Part information and associated storage；
Identity information can be such as name, ID card No. and telephone number.User, can when entering swimming pool
To pass through the photo that face photographing device acquires face in inlet, and identity information is input to the associate device of inlet
On, human face photo and identity information are realized in server associated storage, as pre-registration.It is useful that institute thus may be implemented
The monitoring of family traceability, not only may determine that whether user drowns, but also may determine that whether some specific user drowns.
Extracts static frames from the video of the acquisition at predetermined time intervals in the step s 120.
In this exemplary embodiment, it is made of due to video continuous static frames, it can be every predetermined
Time, such as a static frames are extracted every 5s.Static frames are extracted from collected video to be used in subsequent step
Identify personage, so judge the personage whether generation is drowned, every predetermined time extraction can under the premise of guaranteed efficiency,
The calculation processing load in subsequent step based on video frame is effectively reduced.
Identifies personage from the static frames in step s 130.
In this exemplary embodiment, static frames are exactly to form the video frame of video, include various images in video frame
Information, such as personage, water and swim ring etc. can determine the video comprising personage by identifying personage from video frame
Frame, and then exclude other for judging whether personage occurs drowned to judge useless video frame；Further, known by face
Face can not be identified from video frame accurately, and then the personage is determined by database matching, is realized specific
The drowned situation of the traceability of some personage monitors.
In a kind of this exemplary embodiment, personage is identified from the static frames refering to what is shown in Fig. 3, described, including
Step S310, step S320 and step S330:
Step S310. identifies face from the static frames using recognition of face；
The face identified is compared step S320. with the human face photo of associated storage, obtains comparison result,
To identify personage；
The face that step S330. is such as identified is consistent with the human face photo of associated storage, then the identity information of associated storage
As the identity of personage information identified.
Face can be accurately identified from the video frame comprising face using existing recognition of face, and then will identification
The face of personage out is compared with the human face photo of associated storage, when the similarity of two faces is more than certain threshold value,
Not Chu personage face it is consistent with the human face photo of an associated storage in pre-registration, then it is assumed that have identified this
People, using the identity information of associated storage as the identity of personage information identified.The accurate of target person may be implemented in this way
Monitoring.
If identifies personage from the static frames in step S140, determine that the static frames for identifying personage are
Target frame, the personage recognized are target person.
In this exemplary embodiment, after determining personage in static frames, this determines the video frame of personage just
Be can with personage described in effecting reaction whether An Quan target frame, that is, whether drown, and then can be in subsequent middle use
In the judgement whether target person drowns, the security monitoring of the clear information of each personage in swimming pool is realized, that is, traceable
Property monitoring observation.
For from the static frames continuously extracted after the target frame, determination identifies the target in step S150
The static frames of the predetermined number of personage.
In this exemplary embodiment, firstly, the face for the personage that storage is identified from static frames；It then, will be
The face alignment of the face identified and storage in the static frames extracted after next predetermined time, if unanimously, continued
By the face alignment of the face identified and storage in the static frames extracted after next one predetermined time, until the static state
The face identified in the static frames that continuous predetermined number extracts after frame is all identical as the face of storage.For example, every
5s extracts a static frames.If after the current static frame in continuous 6 static frames extracted, i.e. after 5s, after 10s,
In the static frames extracted after 15s, after 20s, after 25s, after 30s, the face identified is all identical as the face of storage, then identifies
There is the personage in the predetermined number destination frame continuously extracted after the static frames.By being obtained after determining target frame
Multiple frames comprising target person, so that it may which, using variation when drowning with video frame when normal swimming, accurate judgement personage is
It is no to occur to drown, the accuracy rate of judgement is effectively ensured.
The static frames of the predetermined number are inputted trained machine learning model by step S160, are obtained described
The judging result whether target person drowns.
In this exemplary embodiment, drowned view does not occur previously according to the static frames that video when drowning occurs and
The machine learning model of the static frames training of frequency can may be implemented in the swimming video according to predetermined number to include personage's
Static frames are automatic, efficiently and accurately judge whether personnel drown, and then rescue efficiency can be effectively ensured.
In a kind of originally exemplary embodiment, the static frames by the predetermined number input trained engineering
Practise model, comprising:
By the static frame sequence of the predetermined number according to the sequencing of each static frames at the time of point in video, according to
Secondary series connection obtains static frame sequence；
By the trained machine learning model of static frames sequence inputting.
The static frames are connected into static frame sequence, the input efficiency of static frames can be improved, according to the elder generation of moment point
Sequential series afterwards, so that it may which the consistency for guaranteeing static frames and truth effectively improves the accuracy rate of judgement.
In a kind of originally exemplary embodiment, which is characterized in that the training method of the machine learning model is:
Calibration in advance is collected drowned positive sample occurs and the static frame sequence of drowned negative sample does not occur as sample
Collection；
Each static frames sequence samples in sample set are inputted into machine learning model；
If the judging result of machine learning model output and inconsistent to the calibration of sample in advance, adjustment machine learning model
Coefficient, make to control machine learning model when input is positive sample, drowned judging result occurs for output, is negative sample in input
Drowned judging result does not occur for this when, output；
If machine learning model output judging result with it is consistent to the calibration of sample in advance, train terminate, trained
Good machine learning model.
Occur to identify the static frame sequence of personage in drowned video when the personage's swimming acquired in history, be exactly
Occur to extract in the video of drowned situation in history may recognize that the video frame of the predetermined number of drowned personage was connected
Sequence of frames of video, when as positive sample, input machine learning model can be to include personage under effecting reaction really drowned situation
Video frame situation of change, similarly, the various videos that negative sample is exactly personage in swimming, comprising all swimming situations,
But drowned situation does not occur, by positive sample and negative sample training machine learning model, it is trained accurate to be effectively ensured
Property, and then guarantee drowned judgment accuracy whether occurs.
In a kind of originally exemplary embodiment, it includes being arranged around swimming pool that described image, which acquires equipment, for monitoring
The multiple images in difference swimming region acquire equipment, which is characterized in that if described identify personage from the static frames, determine
The static frames for identifying personage are target frame, and the personage recognized is target person, comprising:
The target frame includes multiple target frames from the video obtained in different image capture devices；
The method also includes:
The multiple target frame at the time of point in the video in the target frame source is obtained respectively；
Obtain the location information of the image capture device of the video in the multiple target frame source；
According to the sequencing of the moment point, it is sequentially connected the position of described image acquisition equipment on the electronic map,
Obtain the motion profile of target person.
Personage is in swimming, it will usually have many zone of action, be set by the Image Acquisition in multiple and different regions
It is standby, it is ensured that comprehensive monitoring, in conjunction with according to the location information of the sequencing of target frame and image capture device just
It can be accurately obtained the real-time swimming track of target person, realize accurate monitoring in real time.
In a kind of originally exemplary embodiment, it includes being arranged around swimming pool that described image, which acquires equipment, for monitoring
The multiple images in difference swimming region acquire equipment, which is characterized in that the static frames by the predetermined number input training
Good machine learning model, after obtaining the judging result whether target person drowns, the method also includes:
If the static frames input of the predetermined number in video in one of image capture device is trained
Machine learning model, obtaining the target person is drowned judging result, obtains the static frames source of the predetermined number
Image capture device position as target position；
Obtain the lifeguard position that the electronic equipment that all lifeguards wear reports；
Calculate each lifeguard position to the target position distance；
Assistance requests are issued to the electronic equipment worn apart from the smallest lifeguard, are carried in the assistance requests
The target position.
Due to knowing the position (such as latitude and longitude coordinates) of each image capture device of arrangement in advance, such as store
In the number and position coordinates mapping table of each image capture device.It can be set according to the Image Acquisition of the personage is identified
Standby number searches mapping table, finds corresponding position coordinates.Several lifeguards are arranged in swimming pool, adorn oneself with electronic equipment,
Such as mobile phone.The GPS system at regular intervals of electronic equipment reports lifeguard position to server.In this way, server can be in step
The lifeguard position that the electronic equipment that all lifeguards wear reports is obtained in S170.Due to each lifeguard position, the figure
As acquisition equipment position all obtained, so that it may calculate each lifeguard position to described image acquire equipment position away from
From to include institute in the assistance requests to the electronic equipment sending assistance requests worn apart from the smallest lifeguard
State the position of image capture device.Described apart from the smallest lifeguard is exactly the lifeguard nearest from drowning person, reaches quickly logical
Know that the effect of relief is unfolded in the lifeguard nearest from drowning person, the efficiency of rescue is effectively ensured.
The disclosure additionally provides a kind of device that automatic identification is drowned.Refering to what is shown in Fig. 4, the device that the automatic identification is drowned
It may include video acquiring module 410, extraction module 420, identification module 430, the first determining module 440, the second determining module
450 and judgment module 460.Wherein:
Video acquiring module 410 can be used for receiving the video for the image capture device acquisition arranged around swimming pool；
Extraction module 420 can be used for extracting static frames at predetermined time intervals from the video of the acquisition；
Identification module 430 can be used for identifying personage from the static frames；
First determining module 440 determines the institute for identifying personage if can be used for from the static frames identifying personage
Stating static frames is target frame, and the personage recognized is target person；
Second determining module 450, can be used for from the static frames continuously extracted after the target frame, determine identification
The static frames of the predetermined number of the target person out；
Judgment module 460 can be used for the static frames of the predetermined number inputting trained machine learning model, obtain
The judging result whether drowned to the target person.
The detail of each module is in the side that corresponding automatic identification is drowned in the drowned device of above-mentioned automatic identification
It is described in detail in method, therefore details are not described herein again.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to embodiment of the present disclosure, it is above-described two or more
Module or the feature and function of unit can embody in a module or unit.Conversely, an above-described mould
The feature and function of block or unit can be to be embodied by multiple modules or unit with further division.
In addition, although describing each step of method in the disclosure in the accompanying drawings with particular order, this does not really want
These steps must be executed in this particular order by asking or implying, or having to carry out step shown in whole could realize
Desired result.Additional or alternative, it is convenient to omit multiple steps are merged into a step and executed by certain steps, and/
Or a step is decomposed into execution of multiple steps etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, mobile terminal or network equipment etc.) is executed according to disclosure embodiment
Method.
In an exemplary embodiment of the disclosure, a kind of electronic equipment that can be realized the above method is additionally provided.
Person of ordinary skill in the field it is understood that various aspects of the invention can be implemented as system, method or
Program product.Therefore, various aspects of the invention can be embodied in the following forms, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 500 of this embodiment according to the present invention is described referring to Fig. 5.The electronics that Fig. 5 is shown
Equipment 500 is only an example, should not function to the embodiment of the present invention and use scope bring any restrictions.
As shown in figure 5, electronic equipment 500 is showed in the form of universal computing device.The component of electronic equipment 500 can be with
Including but not limited to: at least one above-mentioned processing unit 510, at least one above-mentioned storage unit 520, the different system components of connection
The bus 530 of (including storage unit 520 and processing unit 510).
Wherein, the storage unit is stored with program code, and said program code can be held by the processing unit 510
Row, so that various according to the present invention described in the execution of the processing unit 510 above-mentioned " illustrative methods " part of this specification
The step of illustrative embodiments.For example, the processing unit 510 can execute step S110 as shown in fig. 1: receiving
The video for the image capture device acquisition arranged around swimming pool；S120: it is mentioned at predetermined time intervals from the video of the acquisition
Take static frames；Step S130: personage is identified from the static frames；Step S140: if identifying personage from the static frames,
It determines and identifies that the static frames of personage are target frame, the personage recognized is target person；Step S150: from described
In the static frames continuously extracted after target frame, the static frames for identifying the predetermined number of the target person are determined；Step
S160: the static frames of the predetermined number are inputted into trained machine learning model, obtain whether the target person drowns
Judging result.
Storage unit 520 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 5201 and/or cache memory unit 5202, it can further include read-only memory unit (ROM) 5203.
Storage unit 520 can also include program/utility with one group of (at least one) program module 5205
5204, such program module 5205 includes but is not limited to: operating system, one or more application program, other program moulds
It may include the realization of network environment in block and program data, each of these examples or certain combination.
Bus 530 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 500 can also be with one or more external equipments 700 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, the equipment that also client can be enabled interact with the electronic equipment 500 with one or more communicates, and/or with make
The electronic equipment 500 any equipment (such as the router, modulatedemodulate that can be communicated with one or more of the other calculating equipment
Adjust device etc.) communication.This communication can be carried out by input/output (I/O) interface 550.Also, electronic equipment 500 may be used also
To pass through network adapter 560 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network
Network, such as internet) communication.As shown, network adapter 560 passes through other modules of bus 530 and electronic equipment 500
Communication.It should be understood that although not shown in the drawings, other hardware and/or software module, packet can be used in conjunction with electronic equipment 500
It includes but is not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, magnetic tape drive
Device and data backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, terminal installation or network equipment etc.) is executed according to disclosure embodiment
Method.
In an exemplary embodiment of the disclosure, a kind of computer readable storage medium is additionally provided, energy is stored thereon with
Enough realize the program product of this specification above method.In some possible embodiments, various aspects of the invention may be used also
In the form of being embodied as a kind of program product comprising program code, when described program product is run on the terminal device, institute
Program code is stated for executing the terminal device described in above-mentioned " illustrative methods " part of this specification according to this hair
The step of bright various illustrative embodiments.
Refering to what is shown in Fig. 6, describing the program product for realizing the above method of embodiment according to the present invention
600, can using portable compact disc read only memory (CD-ROM) and including program code, and can in terminal device,
Such as it is run on PC.However, program product of the invention is without being limited thereto, in this document, readable storage medium storing program for executing can be with
To be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
It is in connection.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or any above combination.The more specific example of readable storage medium storing program for executing is (non exhaustive
List) include: electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM), read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable signal medium can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.
The program code for including on readable medium can transmit with any suitable medium, including but not limited to wirelessly, have
Line, optical cable, RF etc. or above-mentioned any appropriate combination.
The program for executing operation of the present invention can be write with any combination of one or more programming languages
Code, described program design language include object oriented program language-Java, C++ etc., further include conventional
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in client
It calculates and executes in equipment, partly executes on the client device, being executed as an independent software package, partially in client's calculating
Upper side point is executed on a remote computing or is executed in remote computing device or server completely.It is being related to far
Journey calculates in the situation of equipment, and remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to client computing device, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In addition, above-mentioned attached drawing is only the schematic theory of processing included by method according to an exemplary embodiment of the present invention
It is bright, rather than limit purpose.It can be readily appreciated that the time that above-mentioned processing shown in the drawings did not indicated or limited these processing is suitable
Sequence.In addition, be also easy to understand, these processing, which can be, for example either synchronously or asynchronously to be executed in multiple modules.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
His embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Adaptive change follow the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by claim
It points out.
Claims (10)
1. a kind of drowned method of automatic identification characterized by comprising
Receive the video for the image capture device acquisition arranged around swimming pool；
Static frames are extracted at predetermined time intervals from the video of the acquisition；
Personage is identified from the static frames；
If identifying personage from the static frames, determines and identify that the static frames of personage are target frame, it is described to recognize
Personage be target person；
From the static frames continuously extracted after the target frame, the quiet of the predetermined number for identifying the target person is determined
State frame；
The static frames of the predetermined number are inputted into trained machine learning model, obtain what whether the target person drowned
Judging result.
2. the method according to claim 1, wherein being adopted receiving the image capture device arranged around swimming pool
Before the video of collection, the method also includes:
The human face photo for receiving the shooting of swimming pool inlet face photographing device believes the identity of the human face photo and the person of being taken
Cease associated storage.
3. the method according to claim 1, wherein described identify personage from the static frames, comprising:
Face is identified from the static frames using recognition of face；
The face identified is compared with the human face photo of associated storage, comparison result is obtained, to identify personage；
The face such as identified is consistent with the human face photo of associated storage, then using the identity information of associated storage as identifying
Identity of personage information.
4. the method according to claim 1, wherein described train the static frames input of the predetermined number
Machine learning model, comprising:
By the static frame sequence of the predetermined number according to the sequencing of each static frames at the time of point in video, successively go here and there
Connection, obtains static frame sequence；
By the trained machine learning model of static frames sequence inputting.
5. the method according to claim 1, wherein the training method of the machine learning model is:
Calibration in advance is collected drowned positive sample occurs and the static frame sequence of drowned negative sample does not occur as sample set；
Each static frames sequence samples in sample set are inputted into machine learning model；
If the judging result of machine learning model output with it is inconsistent to the calibration of sample in advance, adjust machine learning model is
Number makes to control machine learning model when input is positive sample, and drowned judging result occurs for output, is negative sample in input
When, drowned judging result does not occur for output；
If machine learning model output judging result with it is consistent to the calibration of sample in advance, train terminate, obtain trained
Machine learning model.
6. according to the method described in claim 1, described image acquisition equipment includes being arranged around swimming pool, for monitoring difference
The multiple images in swimming region acquire equipment, which is characterized in that if described identify personage from the static frames, determine identification
The static frames of personage are target frame out, and the personage recognized is target person, comprising:
The target frame includes multiple target frames from the video obtained in different image capture devices；
The method also includes:
The multiple target frame at the time of point in the video in the target frame source is obtained respectively；
Obtain the location information of the image capture device of the video in the multiple target frame source；
According to the sequencing of the moment point, it is sequentially connected the position of described image acquisition equipment on the electronic map, obtains
The motion profile of target person.
7. according to the method described in claim 1, described image acquisition equipment includes being arranged around swimming pool, for monitoring difference
The multiple images in swimming region acquire equipment, which is characterized in that the static frames input by the predetermined number is trained
Machine learning model, after obtaining the judging result whether target person drowns, the method also includes:
If the static frames of the predetermined number in video in one of image capture device are inputted trained machine
Device learning model, obtaining the target person is drowned judging result, obtains the figure in the static frames source of the predetermined number
As the position of acquisition equipment is as target position；
Obtain the lifeguard position that the electronic equipment that all lifeguards wear reports；
Calculate each lifeguard position to the target position distance；
Assistance requests are issued to the electronic equipment worn apart from the smallest lifeguard, are carried in the assistance requests described
Target position.
8. a kind of drowned device of automatic identification characterized by comprising
Video acquiring module receives the video for the image capture device acquisition arranged around swimming pool；
Extraction module extracts static frames from the video of the acquisition at predetermined time intervals；
Identification module identifies personage from the static frames；
First determining module determines if identifying personage from the static frames and identifies that the static frames of personage are target
Frame, the personage recognized are target person；
Second determining module, from the static frames continuously extracted after the target frame, determination identifies the target person
Predetermined number static frames；
The static frames of the predetermined number are inputted trained machine learning model, obtain the target person by judgment module
Whether Ni Shui judging result.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt
Claim 1-7 described in any item methods are realized when processor executes.
10. a kind of electronic equipment characterized by comprising
Memory executes computer-readable instruction for storing；
Processor is required described in any of 1-7 for executing the computer-readable instruction stored in memory with perform claim
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811554491.9A CN109815805A (en) | 2018-12-18 | 2018-12-18 | Automatic identification drowned method, apparatus, storage medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811554491.9A CN109815805A (en) | 2018-12-18 | 2018-12-18 | Automatic identification drowned method, apparatus, storage medium and electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109815805A true CN109815805A (en) | 2019-05-28 |
Family
ID=66602170
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811554491.9A Pending CN109815805A (en) | 2018-12-18 | 2018-12-18 | Automatic identification drowned method, apparatus, storage medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109815805A (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363153A (en) * | 2019-07-16 | 2019-10-22 | 广州图普网络科技有限公司 | It paddles detection method, device, server and computer readable storage medium |
| CN111565259A (en) * | 2019-11-20 | 2020-08-21 | 王涛 | IP data packet wireless sending platform and method |
| CN112165600A (en) * | 2020-08-26 | 2021-01-01 | 苏宁云计算有限公司 | Drowning identification method and device, camera and computer system |
| CN112489371A (en) * | 2020-11-26 | 2021-03-12 | 上海天健体育科技发展有限公司 | Swimming pool drowning prevention early warning system based on computer vision |
| CN113468945A (en) * | 2021-03-26 | 2021-10-01 | 厦门大学 | Swimmer drowning detection method |
| CN113762215A (en) * | 2021-10-12 | 2021-12-07 | 江阴市人人达科技有限公司 | Real-time judgment system for object surrounding environment |
| CN114170317A (en) * | 2022-01-10 | 2022-03-11 | 杭州巨岩欣成科技有限公司 | Method and device for judging position of drowning-proof human head of swimming pool, computer equipment and storage medium thereof |
| CN114582091A (en) * | 2022-03-11 | 2022-06-03 | 佛山市高明粤华卫生洁具有限公司 | Intelligent monitoring and early warning system based on swimming pool safety |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413114A (en) * | 2013-05-17 | 2013-11-27 | 浙江大学 | Near-drowning behavior detection method based on support vector machine |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN106296724A (en) * | 2015-05-12 | 2017-01-04 | 杭州海康威视数字技术股份有限公司 | Determination method, system and the processing server of the trace information of a kind of target person |
| CN106874827A (en) * | 2015-12-14 | 2017-06-20 | 北京奇虎科技有限公司 | Video frequency identifying method and device |
| CN107622505A (en) * | 2017-09-07 | 2018-01-23 | 青岛博晶微电子科技有限公司 | A kind of drowned monitor and detection method of swimming pool |
| CN108181608A (en) * | 2018-02-01 | 2018-06-19 | 杭州水豚科技有限公司 | The intelligent swimming pool quickly positioned prevents drowned monitoring system and method |
| CN108647575A (en) * | 2018-04-10 | 2018-10-12 | 西北工业大学 | Drowned method for early warning based on optical visual analysis |
| CN208079397U (en) * | 2018-04-28 | 2018-11-09 | 武汉工商学院 | A kind of anti-drowned system and device for swimming pool positioning |
-
2018
- 2018-12-18 CN CN201811554491.9A patent/CN109815805A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103413114A (en) * | 2013-05-17 | 2013-11-27 | 浙江大学 | Near-drowning behavior detection method based on support vector machine |
| CN106296724A (en) * | 2015-05-12 | 2017-01-04 | 杭州海康威视数字技术股份有限公司 | Determination method, system and the processing server of the trace information of a kind of target person |
| CN105354543A (en) * | 2015-10-29 | 2016-02-24 | 小米科技有限责任公司 | Video processing method and apparatus |
| CN106874827A (en) * | 2015-12-14 | 2017-06-20 | 北京奇虎科技有限公司 | Video frequency identifying method and device |
| CN107622505A (en) * | 2017-09-07 | 2018-01-23 | 青岛博晶微电子科技有限公司 | A kind of drowned monitor and detection method of swimming pool |
| CN108181608A (en) * | 2018-02-01 | 2018-06-19 | 杭州水豚科技有限公司 | The intelligent swimming pool quickly positioned prevents drowned monitoring system and method |
| CN108647575A (en) * | 2018-04-10 | 2018-10-12 | 西北工业大学 | Drowned method for early warning based on optical visual analysis |
| CN208079397U (en) * | 2018-04-28 | 2018-11-09 | 武汉工商学院 | A kind of anti-drowned system and device for swimming pool positioning |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110363153A (en) * | 2019-07-16 | 2019-10-22 | 广州图普网络科技有限公司 | It paddles detection method, device, server and computer readable storage medium |
| CN111565259A (en) * | 2019-11-20 | 2020-08-21 | 王涛 | IP data packet wireless sending platform and method |
| CN112165600A (en) * | 2020-08-26 | 2021-01-01 | 苏宁云计算有限公司 | Drowning identification method and device, camera and computer system |
| CN112489371A (en) * | 2020-11-26 | 2021-03-12 | 上海天健体育科技发展有限公司 | Swimming pool drowning prevention early warning system based on computer vision |
| CN112489371B (en) * | 2020-11-26 | 2022-09-13 | 上海天健体育科技发展有限公司 | Swimming pool drowning prevention early warning system based on computer vision |
| CN113468945A (en) * | 2021-03-26 | 2021-10-01 | 厦门大学 | Swimmer drowning detection method |
| CN113762215A (en) * | 2021-10-12 | 2021-12-07 | 江阴市人人达科技有限公司 | Real-time judgment system for object surrounding environment |
| CN114170317A (en) * | 2022-01-10 | 2022-03-11 | 杭州巨岩欣成科技有限公司 | Method and device for judging position of drowning-proof human head of swimming pool, computer equipment and storage medium thereof |
| CN114170317B (en) * | 2022-01-10 | 2024-04-05 | 杭州巨岩欣成科技有限公司 | Swimming pool drowning prevention head position judging method and device and computer equipment |
| CN114582091A (en) * | 2022-03-11 | 2022-06-03 | 佛山市高明粤华卫生洁具有限公司 | Intelligent monitoring and early warning system based on swimming pool safety |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109815805A (en) | Automatic identification drowned method, apparatus, storage medium and electronic equipment | |
| US20190065906A1 (en) | Method and apparatus for building human face recognition model, device and computer storage medium | |
| WO2019200781A1 (en) | Receipt recognition method and device, and storage medium | |
| CN106341661B (en) | Patrol robot | |
| US20140176749A1 (en) | Collecting Photos | |
| CN108108711B (en) | Face control method, electronic device and storage medium | |
| CN104143086A (en) | Application technology of portrait comparison to mobile terminal operating system | |
| US20160166204A1 (en) | Detecting visual impairment through normal use of a mobile device | |
| US20210174073A1 (en) | A Method and Apparatus for Swimmer Tracking | |
| WO2014100280A1 (en) | Sharing photos | |
| CN105267013A (en) | Head-wearing intelligent vision obstruction assisting system | |
| CN111783674A (en) | Face recognition method and system based on AR glasses | |
| CN113505704B (en) | Personnel safety detection method, system, equipment and storage medium for image recognition | |
| CN111126411A (en) | Abnormal behavior identification method and device | |
| CN112700568B (en) | Identity authentication method, equipment and computer readable storage medium | |
| CN114724246A (en) | Dangerous behavior identification method and device | |
| EP2833321A1 (en) | Information processing device, information processing method and program | |
| CN103207990A (en) | People recognition system based on mobile terminal and for police | |
| CN113221766A (en) | Method for training living body face recognition model and method for recognizing living body face and related device | |
| CN112861711A (en) | Regional intrusion detection method and device, electronic equipment and storage medium | |
| Zhang et al. | Face mask recognition based on object detection | |
| CN112633244A (en) | Social relationship identification method and device, electronic equipment and storage medium | |
| CN112580560A (en) | Intelligent monitoring method and system applied to swimming pool | |
| CN110825231A (en) | ROV gesture control system | |
| CN113806361B (en) | Method, device and storage medium for associating electronic monitoring equipment with road |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| CB02 | Change of applicant information |
Address after: 201, room 518000, building A, No. 1, front Bay Road, Qianhai Shenzhen Guangdong Shenzhen Hong Kong cooperation zone (Qianhai business secretary) Applicant after: Shenzhen one ledger Intelligent Technology Co., Ltd. Address before: 518000 Guangdong city of Shenzhen province Qianhai Shenzhen Hong Kong cooperation zone before Bay Road No. 1 building 201 room A Applicant before: Shenzhen one ledger Intelligent Technology Co., Ltd. |
|
| CB02 | Change of applicant information | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |