CN107454376B - Fall based reminding method, device, electronic device and computer readable storage medium - Google Patents
Fall based reminding method, device, electronic device and computer readable storage medium Download PDFInfo
- Publication number
- CN107454376B CN107454376B CN201710811958.2A CN201710811958A CN107454376B CN 107454376 B CN107454376 B CN 107454376B CN 201710811958 A CN201710811958 A CN 201710811958A CN 107454376 B CN107454376 B CN 107454376B
- Authority
- CN
- China
- Prior art keywords
- depth
- depth image
- active user
- image
- edge pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Business, Economics & Management (AREA)
- Emergency Management (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses one kind to fall based reminding method, device and computer readable storage medium.Wherein method includes: the first depth image for obtaining active user, and obtains the second depth image of active user place current scene；Edge extracting is carried out to obtain depth data of the edge pixel of each object in the second depth image in the second depth image to the second depth image；According to the depth data of the edge pixel of each object in the first depth image and the second depth image of active user, judge the front of active user in current scene with the presence or absence of step；If so, generating prompting message and being supplied to active user.The embodiment of the present invention can be gone out with accurate calibration plantar foot area sole edge pixel and object area edge pixel further, it whether there is the judgement better effect of step based on the front that more accurately sole edge pixel and the depth data of object edge pixel carry out active user in the current scene, the user experience is improved.
Description
Technical field
The present invention relates to technical field of image processing more particularly to one kind to fall based reminding method, device, electronic device and meter
Calculation machine readable storage medium storing program for executing.
Background technique
In the related technology, when user's walking is had such as step in the background in the virtual environment scene of difference in height, lead to
It is often used feature point extraction character contour and contour of object, and judges current virtual scene according to character contour and contour of object
The front of middle active user whether there is step.But it is this accurate using the character contour and contour of object of feature point extraction
Spend it is not high, especially can not accurate calibration go out the boundary of personage and object, influence in the current virtual scene before active user
Side whether there is the judgement effect of step.
Summary of the invention
The purpose of the present invention is intended to solve above-mentioned one of technical problem at least to a certain extent.
For this purpose, the first purpose of this invention is to propose that one kind falls based reminding method.This method can be gone out with accurate calibration
The edge pixel of the sole edge pixel of plantar foot area and object area further, based on more accurately sole edge pixel
It is imitated with the presence or absence of the judgement of step in the front for carrying out active user in the current scene with the depth data of object edge pixel
More preferably, the user experience is improved for fruit.
Second object of the present invention is to propose that one kind falls alarm set.
Third object of the present invention is to propose a kind of electronic device.
Fourth object of the present invention is to propose a kind of computer readable storage medium.
In order to achieve the above objectives, what first aspect present invention embodiment proposed falls based reminding method, comprising: obtains current use
First depth image at family, and obtain the second depth image of active user place current scene；To second depth
Image carries out edge extracting to obtain the edge pixel of each object in second depth image in second depth image
In depth data；According to the edge of each object in the first depth image of the active user and second depth image
The depth data of pixel judges the front of active user in the current scene with the presence or absence of step；Letter is reminded if so, generating
It ceases and is supplied to the active user.
In order to achieve the above objectives, what second aspect of the present invention embodiment proposed falls alarm set, comprising: depth image is adopted
Collect component, the depth image acquisition component is used to obtain the first depth image of active user, and obtains the active user
Second depth image of place current scene；Processor, the processor are used for: being carried out edge to second depth image and mentioned
It takes to obtain depth data of the edge pixel of each object in second depth image in second depth image；Root
According to the depth data of the edge pixel of each object in the first depth image and second depth image of the active user,
Judge the front of active user in the current scene with the presence or absence of step；If so, generating prompting message and being supplied to described
Active user.
In order to achieve the above objectives, the electronic device that third aspect present invention embodiment proposes, comprising: one or more processing
Device；Memory；With one or more programs, wherein one or more of programs are stored in the memory, and by
It is configured to be executed by one or more of processors, described program includes for executing described in first aspect present invention embodiment
The instruction for falling based reminding method.
In order to achieve the above objectives, fourth aspect present invention embodiment propose computer readable storage medium, including with energy
The computer program that the electronic device enough imaged is used in combination, the computer program can be executed by processor to complete the present invention
Fall based reminding method described in first aspect embodiment.
Embodiment of the present invention falls based reminding method, falls alarm set, electronic device and computer-readable storage medium
Matter is worked as by obtaining the depth data of sole edge pixel from the first depth image of active user, and where active user
The depth data of the object edge pixel of each object is obtained in second depth image of preceding scene, and according to sole edge pixel
To realize, whether active user is moved at step edge in the current scene with the depth data of object edge pixel.Due to
The acquisition of depth image is not easy to be illuminated by the light, in scene therefore the influence of the factors such as COLOR COMPOSITION THROUGH DISTRIBUTION is extracted by depth image
People's plantar foot area and object area are more accurate, it is particularly possible to which accurate calibration goes out sole edge pixel and the object areas of plantar foot area
The edge pixel in domain.Further, based on more accurately sole edge pixel and the depth data of object edge pixel carry out
The front of active user whether there is the judgement better effect of step in the current scene, and the user experience is improved.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partially become from the following description
Obviously, or practice through the invention is recognized.
Detailed description of the invention
In order to more clearly explain the embodiment of the invention or the technical proposal in the existing technology, to embodiment or will show below
There is attached drawing needed in technical description to be briefly described, it should be apparent that, the accompanying drawings in the following description is only this
Some embodiments of invention for those of ordinary skill in the art without creative efforts, can be with
It obtains other drawings based on these drawings.
Fig. 1 is the flow chart for falling based reminding method of certain embodiments of the present invention；
Fig. 2 is the module diagram for falling alarm set of certain embodiments of the present invention；
Fig. 3 is the structural schematic diagram of the electronic device of certain embodiments of the present invention；
Fig. 4 is the flow chart for falling based reminding method of certain embodiments of the present invention；
Fig. 5 is the flow chart for falling based reminding method of certain embodiments of the present invention；
Fig. 6 (a) to Fig. 6 (e) is the schematic diagram of a scenario of structural light measurement according to an embodiment of the invention；
The schematic diagram of a scenario of Fig. 7 (a) and Fig. 7 (b) structural light measurement according to an embodiment of the invention；
Fig. 8 is the flow chart for falling based reminding method of certain embodiments of the present invention；
Fig. 9 is the module diagram of the electronic device of certain embodiments of the present invention；
Figure 10 is the module diagram of the electronic device of certain embodiments of the present invention.
Specific embodiment
In order to make the objectives, technical solutions, and advantages of the present invention clearer, with reference to the accompanying drawings and embodiments, right
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
It is appreciated that term " first " used in the present invention, " second " etc. can be used to describe various elements herein,
But these elements should not be limited by these terms.These terms are only used to distinguish the first element from the other element.Citing comes
It says, without departing from the scope of the invention, the first client can be known as the second client, and similarly, can incite somebody to action
Second client is known as the first client.The first client and the second client both client, but it is not same visitor
Family end.
Below with reference to the accompanying drawings describe the embodiment of the present invention falls based reminding method, device, electronic device and computer-readable
Storage medium.
Also referring to Fig. 1 to 2, the based reminding method that falls of the embodiment of the present invention can be applied to the electronics of the embodiment of the present invention
Device 1000.This falls based reminding method
S110, obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image.
S120 carries out edge extracting to the second depth image to obtain the edge pixel of each object in the second depth image
Depth data in the second depth image.
As an example, edge extracting can be carried out to second depth image by Canny operator.Wherein, Canny
The core that operator carries out the algorithm of edge extracting mainly includes the following steps: firstly, with 2D gaussian filtering template to the second depth
Image carries out convolution to eliminate noise；Then, the gradient value of the gray scale of each pixel is obtained using differential operator, and according to gradient
Value calculates the gradient direction of the gray scale of each pixel, can find adjoining picture of the respective pixel along gradient direction by gradient direction
Element；Then, each pixel is traversed, if the gray scale of the gray value of some pixel and former and later two adjacent pixels on its gradient direction
Value is not compared to being the largest, it is considered that this pixel is not marginal point.In this way, can determine in second depth image
In the pixel of marginal position, to obtain the edge pixel of each object after edge extracting, and then each object can get
Depth data of the edge pixel in second depth image.
S130, according to the depth of the edge pixel of each object in the first depth image and the second depth image of active user
Degree evidence judges the front of active user in current scene with the presence or absence of step.
S140, if so, generating prompting message and being supplied to active user.
Referring to Fig. 3, the embodiment of the present invention falls based reminding method and can fall alarm set by the embodiment of the present invention
100 realize.The alarm set 100 that falls of the embodiment of the present invention can be used for the electronic device 1000 of the embodiment of the present invention.Fall and mentions
Device 100 of waking up may include depth image acquisition component 10 and processor 20.Above-mentioned steps S110 can acquire group by depth image
Part 10 realizes that step S120~step S140 can be realized by processor 20.
That is, depth image acquisition component 10 can be used for obtaining the first depth image of active user, and obtain institute
Second depth image of current scene where stating active user；Processor 20 can be used for carrying out edge to second depth image
It extracts to obtain depth data of the edge pixel of each object in second depth image in second depth image,
And according to the depth data of the first depth image of the active user and the edge pixel of each object, work as described in judgement
The front of active user whether there is step in preceding scene, if so, generating prompting message and being supplied to the active user.
Wherein, the second depth image characterization includes the depth information of each people or object in the background of active user,
The first depth image characterization of the active user includes the depth information of personage in the background of the active user.Described second
Each pixel in depth image can find the depth information of the corresponding pixel in first depth image.
The alarm set 100 that falls of the embodiment of the present invention can be applied to the electronic device 1000 of the embodiment of the present invention.?
That is the electronic device 1000 of the embodiment of the present invention may include that the embodiment of the present invention falls alarm set 100.
In some embodiments, which can have shooting function, and the shooting function is to utilize knot
The shooting of structure light principle progress depth image.For example, the electronic device 1000 can be smart phone, platform computer, intelligent head
Helmet, intelligent glasses etc.；It can also be VR (Virtual Reality, virtual reality technology) equipment, AR (Augmented
Reality, augmented reality) equipment etc..
Due to the acquisition of depth image is not easy to be illuminated by the light, in scene the factors such as COLOR COMPOSITION THROUGH DISTRIBUTION influence, and in depth image
The depth information for being included has higher accuracy, therefore, the personage edge that is extracted by the first depth image and logical
Cross the object edge that the second depth image extracts, more accurately, it is particularly possible to accurate calibration go out personage's sole edge pixel and
Object edge pixel.Further, based on the more accurately depth data of personage's sole edge pixel and object edge pixel
The front for carrying out active user in current scene whether there is the judgement better effect of step.
As an example, referring to Fig. 4, in some embodiments, above-mentioned steps S110 obtains the of active user
The step of one depth image can include:
S1101, to active user's projective structure light；
S1102 shoots the structure light image modulated through active user；
S1103, the corresponding phase information of each pixel of demodulation structure light image is to obtain the first depth image.
Referring again to Fig. 3, in some embodiments, depth image acquisition component 10 may include 11 He of structured light projector
Structure light video camera head 12.Step S1101 can be realized that step S1102 and step S1103 can be by tying by structured light projector 11
Structure light video camera head 12 is realized.
That is, structured light projector 11 can be used for active user's projective structure light；Structure light video camera head 12 can be used
In the structure light image that shooting is modulated through the active user, and demodulate the corresponding phase of each pixel of the structure light image
Information is to obtain first depth image.
For example, structured light projector 11 can be by the face and body of the project structured light of certain pattern to active user
After upper, it will form in the face of active user and the surface of body by the modulated structure light image of active user.Structure light is taken the photograph
As first 12 shoot it is modulated after structure light image, then structure light image is demodulated deep to obtain the first of the active user
Spend image.Wherein, the mode of structure light can be laser stripe, Gray code, sine streak, non-homogeneous speckle etc..
Referring to Fig. 5, in some embodiments, the corresponding phase of each pixel of step S1103 demodulation structure light image
The step of position information is to obtain the first depth image can include:
S11031, the corresponding phase information of each pixel in demodulation structure light image；
Phase information is converted depth information by S11032；
S11033 generates the first depth image of the active user according to depth information.
Referring again to Fig. 2, in some embodiments, step S11031, step S11032 and step S11033 can be with
It is realized by structure light video camera head 12.
That is, structure light video camera head 12 can be further used for the corresponding phase of each pixel in demodulation structure light image
Information converts depth information for the phase information, and generates first depth image according to the depth information.
For example, compared with non-modulated structure light, the phase information of modulated structure light is changed,
The structure light showed in structure light image is the structure light produced after distortion, wherein the phase information of variation can table
Levy the depth information of object.Therefore, structure light video camera head 12 demodulates the corresponding phase of each pixel in structure light image first
Information calculates depth information further according to phase information, to obtain the first final depth image.
The face and body of active user are acquired in order to be apparent from those skilled in the art more according to structure
The process of first depth image of body is explained by taking a kind of widely used optical grating projection technology (fringe projection technology) as an example below
State its concrete principle.Wherein, optical grating projection technology belongs to sensu lato area-structure light.
As shown in Fig. 6 (a), when using face structured light projection, sine streak is generated by computer programming first,
And sine streak is projected to measured object by structured light projector 11, recycle structure light video camera head 12 to shoot striped by object
Modulated bending degree then demodulates the curved stripes and obtains phase, then converts depth information for phase to obtain
One depth image.To avoid the problem that generating error or error coupler, need before carrying out depth information acquisition using structure light to depth
It spends image collection assembly 10 and carries out parameter calibration, calibration includes geometric parameter (for example, structure light video camera head 12 and project structured light
Relative position parameter between device 11 etc.) calibration, structure light video camera head 12 inner parameter and structured light projector 11
The calibration etc. of inner parameter.
Specifically, the first step, computer programming generates sine streak.It needs to obtain using the striped of distortion due to subsequent
Phase, for example phase is obtained using four step phase-shifting methods, therefore generate four width phase differences here and beStriped, then structure light throw
Emitter 11 projects the four spokes line timesharing on measured object (mask as shown in Fig. 6 (a)), and structure light video camera head 12 collects
Such as the figure on the left side Fig. 6 (b), while the striped of the plane of reference shown in reading on the right of Fig. 6 (b).
Second step carries out phase recovery.Structure light video camera head 12 (is tied according to the bar graph that collected four width is modulated
Structure light image) calculate the phase diagram obtained at this time by phase modulation be truncation phase diagram.Because what four step Phase-shifting algorithms obtained
The result is that calculating gained by arctan function, therefore the modulated phase of structure light is limited between [- π, π], that is to say, that
Whenever modulated phase is more than [- π, π], can restart again.Shown in finally obtained phase main value such as Fig. 6 (c).
Wherein, during carrying out phase recovery, the jump processing that disappears is carried out, i.e., is continuous phase by truncation phase recovery
Position.As shown in Fig. 6 (d), the left side is the continuous phase bitmap modulated, and the right is with reference to continuous phase bitmap.
Third step subtracts each other to obtain phase difference (i.e. phase information) by the continuous phase modulated and with reference to continuous phase, should
Phase difference characterizes depth information of the measured object with respect to the plane of reference, then phase difference is substituted into conversion formula (its of phase and depth
In, the parameter being related in formula is by calibration), the threedimensional model of the object under test as shown in Fig. 6 (e) can be obtained.
It should be understood that in practical applications, according to the difference of concrete application scene, employed in the embodiment of the present invention
Structure light other than above-mentioned grating, can also be any other pattern.
As a kind of possible implementation, the depth information of pattern light progress active user is also can be used in the present invention
Acquisition.
Specifically, the method that pattern light obtains depth information is the diffraction element for being essentially plate using one, this spreads out
The relief diffraction structure that element has particular phases distribution is penetrated, cross section is the step embossment knot with two or more bumps
Structure.Substantially 1 micron of the thickness of substrate in diffraction element, each step it is highly non-uniform, the value range of height can be 0.7
Micron~0.9 micron.Structure shown in Fig. 7 (a) is the local diffraction structure of the collimation beam splitting element of the present embodiment.Fig. 7 (b) is edge
The unit of the cross sectional side view of section A-A, abscissa and ordinate is micron.The speckle pattern of pattern photogenerated has
The randomness of height, and can with the difference of distance changing patterns.Therefore, depth information is being obtained using pattern light
Before, it is necessary first to the speckle pattern in space is calibrated, for example, in the range of 0~4 meter of distance structure light video camera head 12,
As soon as taking a reference planes every 1 centimetre, then 400 width speckle images are saved after demarcating, the spacing of calibration is smaller, obtains
Depth information precision it is higher.Then, structured light projector 11 is by pattern light projection to measured object (such as active user)
On, the speckle pattern that the difference in height on measured object surface to project the pattern light on measured object changes.Structure light
After camera 12 shoots the speckle pattern (i.e. structure light image) projected on measured object, then will be after speckle pattern and calibration early period
The 400 width speckle images saved carry out computing cross-correlation one by one, and then obtain 400 width correlation chart pictures.Testee in space
The position at place can show peak value on correlation chart picture, above-mentioned peak value is superimposed and after interpolation arithmetic
Obtain the depth information of measured object.
Most diffraction lights, but every beam diffraction light light intensity difference are obtained after carrying out diffraction to light beam due to common diffraction element
Greatly, also big to the risk of human eye injury.Re-diffraction even is carried out to diffraction light, the uniformity of obtained light beam is relatively low.
Therefore, the effect projected using the light beam of common diffraction element diffraction to measured object is poor.Using collimation in the present embodiment
Beam splitting element, the element not only have the function of collimating uncollimated rays, also have the function of light splitting, i.e., through reflecting mirror
The non-collimated light of reflection is emitted multi-beam collimation light beam, and the multi-beam collimation being emitted toward different angles after collimating beam splitting element
The area of section of light beam is approximately equal, and flux of energy is approximately equal, so that being carried out using the scatterplot light after the beam diffraction
The effect of projection is more preferable.Meanwhile laser emitting light is dispersed to every light beam, further reduced the risk of injury human eye, and dissipate
Spot structure light is arranged relative to other for uniform structure light, when reaching same collection effect, the consumption of pattern light
Electricity is lower.
It should also be noted that, the implementation of the first depth image of above-mentioned acquisition active user, is also applied for above-mentioned
The acquisition of second depth image of current scene, the description of the acquisition modes of second depth image can be found in above-mentioned current
The description of the acquisition modes of the first depth image of user, details are not described herein.Referring to Fig. 8, in some embodiments, step
Rapid S130 according to the depth data of the edge pixel of each object in the first depth image and the second depth image of active user,
Judge the step of front of active user in current scene is with the presence or absence of step can include:
S1301 carries out personage's edge extracting to the first depth image of active user to determine that the first of active user is deep
Spend the pixel of the marginal position object of personage's sole in image；
S1302 obtains depth data of the sole edge pixel in the first depth image；
S1303, according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data judges the front of active user in current scene with the presence or absence of step.
It as an example, can be according to the side of each object in the depth data and the second depth image of sole edge pixel
Whether the depth data of edge pixel judges the target object edge pixel in sole edge pixel and the second depth image in level
There are differences in height on direction, if so, can determine that there are steps in front of active user in the current scene, otherwise determine institute
Step is not present in the front for stating active user in current scene.
Referring again to Fig. 2, in some embodiments, step S1301, step S1302 and step S1303 can be by
Device 20 is managed to realize.
That is, processor 20 can be further used for carrying out personage's edge extracting to the first depth image of active user
The corresponding pixel of marginal position of personage in the first depth image to determine active user, and obtain sole edge pixel and working as
Depth data in the first depth image of preceding user, and according to object edge in sole edge pixel and the current scene
The depth data of pixel judges the front of active user in the current scene with the presence or absence of step, wherein when sole edge picture
When element has difference in height with the target object edge pixel in the second depth image in the horizontal direction, it can determine that and described work as front court
There are steps in front of active user in scape.
As an example, processor 20 can be carried out by the first depth image of the Canny operator to the active user
Edge extracting.Wherein, the core that Canny operator carries out the algorithm of edge extracting mainly includes the following steps: firstly, with 2D Gauss
Filtering Template carries out convolution to the first depth image of active user to eliminate noise；Then, it is obtained using differential operator each
The gradient value of the gray scale of pixel, and calculate according to gradient value the gradient direction of the gray scale of each pixel, can be with by gradient direction
Find adjacent pixels of the respective pixel along gradient direction；Then, each pixel is traversed, if the gray value of some pixel and its ladder
The gray value of former and later two adjacent pixels is not compared to being the largest on degree direction, it is considered that this pixel is not marginal point.Such as
This, that is, can determine the pixel that marginal position is in the first depth image of the active user, to obtain edge extracting
The edge pixel of personage afterwards, and then can get depth data of the edge pixel of personage in first depth image.
Processor 20, can be according to described when obtaining depth data of the sole edge pixel in first depth image
The depth data of sole edge pixel and object edge pixel, to judge that there are platforms in front of active user in the current scene
Rank.For example, when active user's walking is in the environment scene with difference in height such as step, due to the pixel with difference in height
It is discontinuous in the horizontal direction, it is possible to the characteristic is utilized, to judge whether active user marches to step edge, if
The active user marches at step edge, then can remind the active user.
Processor 20 in determining the current scene in front of active user there are when step, produce prompting message simultaneously
It is supplied to active user.As an example, the prompting message can be supplied to by one or more of mode described
Active user: background side in the mode and change display device of mode, vibrating alert that the mode of voice broadcast, text show
The mode etc. of edge color.
That is, processor 20 when generating prompting message, can be believed the prompting by one or more of mode
Breath is supplied to active user, to remind active user's " there are steps in front, please be careful ", for example, can be shaken by mobile phone
Mode reminds user front that there are steps, alternatively, being reminded in front of user in such a way that voice plays, there are steps, alternatively, logical
It crosses the mode that text shows to show the text in a display device, to remind, there are steps in front of user, alternatively, passing through change
There are steps in front of user to remind for the mode of background edge color in display device.
Also referring to Fig. 3 and Fig. 9, the embodiment of the present invention also proposed a kind of electronic device 1000.Electronic device 1000
It may include falling alarm set 100.Falling alarm set 100 can use hardware and/or software realization.Fall alarm set
100 may include depth image acquisition component 10 and processor 20.
Specifically, depth image acquisition component 10 may include structured light projector 11 and structure light video camera head 12, depth map
As acquisition component 10 can be used for capturing the depth information of active user to obtain the first depth image of the active user, and catch
The depth information of current scene is where catching the active user to obtain the second depth image.For example, with depth image acquisition group
Part 10 captures the depth information of active user for obtaining the first depth image, and structured light projector 11 can be used for structure light
It is projected to active user, wherein structured light patterns can be dissipating for laser stripe, Gray code, sine streak or random alignment
Spot pattern etc..Structure light video camera head 12 includes imaging sensor 121 and lens 122, and the number of lens 122 can be one or more
It is a.Imaging sensor 121 can be used for the structure light image that capturing structure light projector 11 is projected on active user.Structure light figure
As can be sent to by depth image acquisition component 10 processor 20 demodulated, phase recovery, phase information calculate etc. processing with
Obtain the depth information of active user.It is appreciated that the embodiment of the depth information of the current scene can be found in above-mentioned work as
The embodiment of the depth information of preceding user, details are not described herein.
In some embodiments, falling alarm set 100 may include imaging device 110, which may include
Depth image acquisition component 10 and visible image capturing head 111.Wherein, which can be used for capturing made thing body
Colour information to obtain color image.The function of the visible image capturing head 111 and structure light video camera head 12 can be imaged by one
Head is realized, that is to say, that imaging device 10 only includes a camera and a structured light projector 11, and above-mentioned camera is not only
It can shoot coloured image, structure light image can also be shot.
In addition to using structure light obtain depth image other than, can also by binocular vision method, be based on differential time of flight (Time
Of Flight, TOF) even depth obtained as acquisition methods active user the first depth image and current scene it is second deep
Spend image.
Processor 20 further carries out edge extracting to second depth image to obtain in second depth image
Depth data of the edge pixel of each object in second depth image, and to the first depth map of the active user
As carrying out personage's edge extracting with the corresponding pixel of marginal position of personage in determination first depth image, and obtain sole
Depth data of the edge pixel in first depth image, when detecting the sole edge pixel and target object edge
When pixel target object edge pixel, it can determine that there are steps in front of active user in the current scene, at this point, producing
Prompting message is simultaneously supplied to the active user.
In addition, falling alarm set 100 further includes video memory 30.Video memory 30 can be embedded in electronic device
In 1000, it is also possible to independently of the memory outside electronic device 1000, and may include direct memory access (DMA) (Direct
Memory Access, English abbreviation DMA) feature.The raw image data or depth image that visible image capturing head 111 acquires
The structure light image related data that acquisition component 10 acquires, which can transmit, to be stored or is cached into video memory 30.Processing
Device 20 can read structure light image related data to be handled to obtain the first of the active user from video memory 30
Second depth image of depth image and the current scene.In addition, the first depth image of the active user and described working as
Second depth image of preceding scene is also storable in video memory 30, and calling is handled device 20 for processing at any time, for example, place
Reason device 20 call the active user the first depth image carry out personage's edge extracting, and call second depth image into
The edge extracting of each object in row scene.It wherein, can be by personage's sole edge pixel of acquisition and sole edge pixel
Depth data is stored in video memory 30, can also be by object edge pixel in second depth of acquisition and described
The depth data of object edge pixel is stored in video memory 30.
Falling alarm set 100 may also include display 50.Display 50 can obtain prompting letter directly from processor 20
Breath.Display 50 shows the prompting message, and to remind, there are steps in front of active user in the current scene, please be careful.Fall
Falling alarm set 100 may also include encoder/decoder 60, encoder/decoder 60 can active user described in encoding and decoding
The image data of one depth image and the second depth image of the current scene.The image data of coding, which can be saved, to be schemed
As in memory 30, and can before image is shown on display 50 by decoder decompresses to be shown.Encoder/
Decoder 60 can be realized by central processing unit (Central Processing Unit, CPU), GPU or coprocessor.In other words,
Encoder/decoder 60 can be in central processing unit (Central Processing Unit, CPU), GPU and coprocessor
Any one or more.
Falling alarm set 100 further includes control logic device 40.Imaging device 10 imaging when, processor 20 can according at
As the data that equipment obtains are analyzed one or more control parameters (for example, time for exposure etc.) to determine imaging device 10
Image statistics.Image statistics are sent to control logic device 40, the control imaging of control logic device 40 by processor 20
Equipment 10 is imaged with the control parameter determined.Control logic device 40 may include executing one or more routines (such as firmware)
Processor and/or microcontroller.One or more routines can determine imaging device 10 by image statistics based on the received
Control parameter.
Referring to Fig. 10, the electronic device 1000 of the embodiment of the present invention may include one or more processors 200, memory
300 and one or more programs 310.Wherein one or more programs 310 are stored in memory 300, and are configured to
It is executed by one or more processors 200.Program 310 includes falling prompting side for execute above-mentioned any one embodiment
The instruction of method.
For example, program 310 may include for executing the instruction for falling based reminding method described in following steps:
S110 ', obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image；
S120 ' carries out edge extracting to the second depth image to obtain the edge picture of each object in the second depth image
Depth data of the element in the second depth image；
S130 ', according to the edge pixel of each object in the first depth image and the second depth image of active user
Depth data judges the front of active user in current scene with the presence or absence of step；
S140 ', if so, generating prompting message and being supplied to active user.
For another example program 310 further includes for executing the instruction for falling based reminding method described in following steps:
S11031 ', the corresponding phase information of each pixel in demodulation structure light image；
Phase information is converted depth information by S11032 '；
S11033 ' generates the first depth image according to depth information.
In another example program 310 further includes for executing the instruction for falling based reminding method described in following steps:
S1301 ' carries out personage's edge extracting to the first depth image of active user to determine that the first of active user is deep
Spend the pixel of the marginal position object of personage's sole in image；
S1302 ' obtains depth data of the sole edge pixel in the first depth image；
S1303 ', according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data judges the front of active user in current scene with the presence or absence of step.
As an example, according to the edge of each object in the depth data of sole edge pixel and the second depth image
Whether the depth data of pixel judges the target object edge pixel in sole edge pixel and the second depth image in level side
There are differences in height upwards, if so, determining that there are steps in front of active user in current scene.
The computer readable storage medium of embodiment of the present invention includes being combined to make with the electronic device 1000 that can be imaged
Computer program.Computer program can be executed by processor 200 and be mentioned with completing above-mentioned falling for any one embodiment
Awake method.
For example, computer program can be executed by processor 200 to complete to fall based reminding method described in following steps:
S110 ', obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image；
S120 ' carries out edge extracting to the second depth image to obtain the edge picture of each object in the second depth image
Depth data of the element in the second depth image；
S130 ', according to the edge pixel of each object in the first depth image and the second depth image of active user
Depth data judges the front of active user in current scene with the presence or absence of step；
S140 ', if so, generating prompting message and being supplied to active user.
For another example computer program can be executed by processor 200 to complete to fall based reminding method described in following steps:
S11031 ', the corresponding phase information of each pixel in demodulation structure light image；
Phase information is converted depth information by S11032 '；
S11033 ' generates the first depth image according to depth information.
In another example computer program can be executed by processor 200 to complete to fall based reminding method described in following steps:
S1301 ' carries out personage's edge extracting to the first depth image of active user to determine that the first of active user is deep
Spend the pixel of the marginal position object of personage's sole in image；
S1302 ' obtains depth data of the sole edge pixel in the first depth image；
S1303 ', according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data judges the front of active user in current scene with the presence or absence of step.
As an example, according to the edge of each object in the depth data of sole edge pixel and the second depth image
Whether the depth data of pixel judges the target object edge pixel in sole edge pixel and the second depth image in level side
There are differences in height upwards, if so, determining that there are steps in front of active user in current scene.
To sum up, embodiment of the present invention falls based reminding method, alarm set, electronic device and computer-readable storage medium
Matter is worked as by obtaining the depth data of sole edge pixel from the first depth image of active user, and where active user
The depth data of the object edge pixel of each object is obtained in second depth image of preceding scene, and according to sole edge pixel
To realize, whether active user is moved at step edge in the current scene with the depth data of object edge pixel.Due to
The acquisition of depth image is not easy to be illuminated by the light, in scene therefore the influence of the factors such as COLOR COMPOSITION THROUGH DISTRIBUTION is extracted by depth image
People's plantar foot area and object area are more accurate, it is particularly possible to which accurate calibration goes out sole edge pixel and the object areas of plantar foot area
The edge pixel in domain.Further, based on more accurately sole edge pixel and the depth data of object edge pixel carry out
The front of active user whether there is the judgement better effect of step in the current scene, and the user experience is improved.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or spy described in conjunction with this embodiment or example
Point is included at least one embodiment or example of the invention.In the present specification, schematic expression of the above terms are not
It must be directed to identical embodiment or example.Moreover, particular features, structures, materials, or characteristics described can be in office
It can be combined in any suitable manner in one or more embodiment or examples.In addition, without conflicting with each other, the skill of this field
Art personnel can tie the feature of different embodiments or examples described in this specification and different embodiments or examples
It closes and combines.
In addition, term " first ", " second " are used for descriptive purposes only and cannot be understood as indicating or suggesting relative importance
Or implicitly indicate the quantity of indicated technical characteristic.Define " first " as a result, the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the present invention, the meaning of " plurality " is at least two, such as two, three
It is a etc., unless otherwise specifically defined.
Any process described otherwise above or method description are construed as in flow chart or herein, and expression includes
It is one or more for realizing specific logical function or process the step of executable instruction code module, segment or portion
Point, and the range of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discussed suitable
Sequence, including according to related function by it is basic simultaneously in the way of or in the opposite order, Lai Zhihang function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system, including the system of processor or other can be held from instruction
The instruction fetch of row system, device or equipment and the system executed instruction) it uses, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium ", which can be, any may include, stores, communicates, propagates or pass
Defeated program is for instruction execution system, device or equipment or the dress used in conjunction with these instruction execution systems, device or equipment
It sets.The more specific example (non-exhaustive list) of computer-readable medium include the following: there is the electricity of one or more wirings
Interconnecting piece (electronic device), portable computer diskette box (magnetic device), random access memory (RAM), read-only memory
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium can even is that the paper that can print described program on it or other are suitable
Medium, because can then be edited, be interpreted or when necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each section of the invention can be realized with hardware, software, firmware or their combination.Above-mentioned
In embodiment, software that multiple steps or method can be executed in memory and by suitable instruction execution system with storage
Or firmware is realized.It, and in another embodiment, can be under well known in the art for example, if realized with hardware
Any one of column technology or their combination are realized: having a logic gates for realizing logic function to data-signal
Discrete logic, with suitable combinational logic gate circuit specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are understood that realize all or part of step that above-described embodiment method carries
It suddenly is that relevant hardware can be instructed to complete by program, the program can store in a kind of computer-readable storage medium
In matter, which when being executed, includes the steps that one or a combination set of embodiment of the method.
It, can also be in addition, each functional unit in each embodiment of the present invention can integrate in a processing module
It is that each unit physically exists alone, can also be integrated in two or more units in a module.Above-mentioned integrated mould
Block both can take the form of hardware realization, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized and when sold or used as an independent product in the form of software function module, also can store in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only memory, disk or CD etc..Although having been shown and retouching above
The embodiment of the present invention is stated, it is to be understood that above-described embodiment is exemplary, and should not be understood as to limit of the invention
System, those skilled in the art can be changed above-described embodiment, modify, replace and become within the scope of the invention
Type.
Claims (14)
1. one kind falls based reminding method, which comprises the following steps:
The first depth image of active user is obtained, and obtains the second depth image of active user place current scene,
Wherein, each pixel in second depth image can find the depth of the corresponding pixel in first depth image
Information；
Edge extracting is carried out to obtain the edge pixel of each object in second depth image to second depth image
Depth data in second depth image；
According to the depth of the edge pixel of each object in the first depth image of the active user and second depth image
Degree evidence judges the front of active user in the current scene with the presence or absence of step；
If so, generating prompting message and being supplied to the active user；
Wherein, first depth image for obtaining active user, comprising:
Sine streak is generated by computer programming, and the sine streak is projected to measured object by structured light projector,
It recycles structure light video camera head shooting striped by the modulated bending degree of object, then demodulates curved stripes and obtain phase, then
First depth image can be obtained by converting depth information for the phase；Wherein, depth letter is being carried out using structure light
Breath acquisition before, need to depth image acquisition component carry out parameter calibration, it is described calibration include the structure light video camera head with it is described
The calibration of relative position parameter between structured light projector, the inner parameter of the structure light video camera head and the structure light
The calibration of the inner parameter of the projector.
2. falling based reminding method as described in claim 1, which is characterized in that first depth map for obtaining active user
Picture, comprising:
To active user's projective structure light；
Shoot the structure light image modulated through the active user；
The corresponding phase information of each pixel of the structure light image is demodulated to obtain first depth image.
3. falling based reminding method as claimed in claim 2, which is characterized in that each picture of the demodulation structure light image
The corresponding phase information of element is to obtain first depth image, comprising:
Demodulate the corresponding phase information of each pixel in the structure light image；
Depth information is converted by the phase information；
First depth image is generated according to the depth information.
4. falling based reminding method as described in claim 1, which is characterized in that first depth according to the active user
The depth data of the edge pixel of each object, judges currently to use in the current scene in image and second depth image
The front at family whether there is step, comprising:
Personage's edge extracting is carried out with the first depth of the determination active user to the first depth image of the active user
The pixel of the marginal position object of personage's sole in image；
Obtain depth data of the sole edge pixel in first depth image；
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degree evidence judges the front of active user in the current scene with the presence or absence of step.
5. falling based reminding method as claimed in claim 4, which is characterized in that the depth according to the sole edge pixel
The depth data of the edge pixel of each object, judges currently to use in the current scene in data and second depth image
The front at family whether there is step, comprising:
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Whether degree evidence judges the target object edge pixel in the sole edge pixel and second depth image in level side
There are differences in height upwards；
If so, determining that there are steps in front of active user in the current scene.
6. falling based reminding method as described in any one of claims 1 to 5, which is characterized in that pass through one or more of
The prompting message is supplied to the active user by mode:
Background edge color in the mode and change display device of mode, vibrating alert that the mode of voice broadcast, text show
Mode.
7. one kind falls alarm set characterized by comprising
Depth image acquisition component, the depth image acquisition component are used to obtain the first depth image of active user, and obtain
Second depth image of current scene where taking the active user, wherein each pixel in second depth image is equal
The depth information of the corresponding pixel can be found in first depth image；
Processor, the processor are used for:
Edge extracting is carried out to obtain the edge pixel of each object in second depth image to second depth image
Depth data in second depth image；
According to the depth of the edge pixel of each object in the first depth image of the active user and second depth image
Degree evidence judges the front of active user in the current scene with the presence or absence of step；
If so, generating prompting message and being supplied to the active user；
Wherein, first depth image for obtaining active user, comprising:
Sine streak is generated by computer programming, and the sine streak is projected to measured object by structured light projector,
It recycles structure light video camera head shooting striped by the modulated bending degree of object, then demodulates curved stripes and obtain phase, then
First depth image can be obtained by converting depth information for the phase；Wherein, depth letter is being carried out using structure light
Breath acquisition before, need to depth image acquisition component carry out parameter calibration, it is described calibration include the structure light video camera head with it is described
The calibration of relative position parameter between structured light projector, the inner parameter of the structure light video camera head and the structure light
The calibration of the inner parameter of the projector.
8. falling alarm set as claimed in claim 7, which is characterized in that the depth image acquisition component includes structure light
The projector and structure light video camera head, the structured light projector are used for active user's projective structure light；
The structure light video camera head is used for:
Shoot the structure light image modulated through the active user；
The corresponding phase information of each pixel of the structure light image is demodulated to obtain first depth image.
9. falling alarm set as claimed in claim 8, which is characterized in that the structure light video camera head is specifically used for:
Demodulate the corresponding phase information of each pixel in the structure light image；
Depth information is converted by the phase information；
First depth image is generated according to the depth information.
10. falling alarm set as claimed in claim 7, which is characterized in that the processor is specifically used for:
Personage's edge extracting is carried out with the first depth of the determination active user to the first depth image of the active user
The pixel of the marginal position object of personage's sole in image；
Obtain depth data of the sole edge pixel in first depth image；
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degree evidence judges the front of active user in the current scene with the presence or absence of step.
11. falling alarm set as claimed in claim 10, which is characterized in that the processor is specifically used for:
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Whether degree evidence judges the target object edge pixel in the sole edge pixel and second depth image in level side
There are differences in height upwards；
If so, determining that there are steps in front of active user in the current scene.
12. falling alarm set as described in any one of claim 7 to 11, which is characterized in that by following a kind of or more
The prompting message is supplied to the active user by kind mode:
Background edge color in the mode and change display device of mode, vibrating alert that the mode of voice broadcast, text show
Mode.
13. a kind of electronic device, which is characterized in that the electronic device includes:
One or more processors；
Memory；With
One or more programs, wherein one or more of programs are stored in the memory, and be configured to by
One or more of processors execute, and described program includes mentioning for falling described in any one of perform claim requirement 1 to 6
The instruction for method of waking up.
14. a kind of computer readable storage medium, which is characterized in that including the meter being used in combination with the electronic device that can be imaged
Calculation machine program, the computer program can be executed by processor to complete described in any one of claims 1 to 6 to fall prompting
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710811958.2A CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710811958.2A CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107454376A CN107454376A (en) | 2017-12-08 |
| CN107454376B true CN107454376B (en) | 2019-07-19 |
Family
ID=60495300
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710811958.2A Active CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107454376B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110266891A (en) * | 2019-06-28 | 2019-09-20 | 无锡睿勤科技有限公司 | Information cuing method, device, terminal and storage medium |
| CN111899348A (en) * | 2020-07-14 | 2020-11-06 | 四川深瑞视科技有限公司 | Projection-based augmented reality experiment demonstration system and method |
| CN112535476B (en) * | 2020-12-01 | 2022-11-22 | 业成科技（成都）有限公司 | Fall detection system and method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104680525A (en) * | 2015-02-12 | 2015-06-03 | 南通大学 | Automatic human body fall-down detection method based on Kinect depth image |
| CN105934227A (en) * | 2014-01-24 | 2016-09-07 | 微软技术许可有限责任公司 | Audio navigation assistance |
| CN106126042A (en) * | 2016-07-01 | 2016-11-16 | 京东方科技集团股份有限公司 | A kind of environment based reminding method and system |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9671566B2 (en) * | 2012-06-11 | 2017-06-06 | Magic Leap, Inc. | Planar waveguide apparatus with diffraction element(s) and system employing same |
-
2017
- 2017-09-11 CN CN201710811958.2A patent/CN107454376B/en active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105934227A (en) * | 2014-01-24 | 2016-09-07 | 微软技术许可有限责任公司 | Audio navigation assistance |
| CN104680525A (en) * | 2015-02-12 | 2015-06-03 | 南通大学 | Automatic human body fall-down detection method based on Kinect depth image |
| CN106126042A (en) * | 2016-07-01 | 2016-11-16 | 京东方科技集团股份有限公司 | A kind of environment based reminding method and system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107454376A (en) | 2017-12-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107682607B (en) | Image acquiring method, device, mobile terminal and storage medium | |
| CN107734267B (en) | Image processing method and device | |
| CN107610077A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107483845B (en) | Photographic method and its device | |
| CN107465906B (en) | Panorama shooting method, device and the terminal device of scene | |
| CN107742296A (en) | Dynamic image generation method and electronic installation | |
| CN107452034B (en) | Image processing method and device | |
| CN107797664A (en) | Content display method, device and electronic installation | |
| CN107995434A (en) | Image acquiring method, electronic device and computer-readable recording medium | |
| CN107734264B (en) | Image processing method and device | |
| CN107509045A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107707831A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107742300A (en) | Image processing method, device, electronic installation and computer-readable recording medium | |
| CN107610127B (en) | Image processing method, image processing apparatus, electronic apparatus, and computer-readable storage medium | |
| CN107807806A (en) | Display parameters method of adjustment, device and electronic installation | |
| CN107590828B (en) | Blurring processing method and device for shot image | |
| CN107454376B (en) | Fall based reminding method, device, electronic device and computer readable storage medium | |
| WO2019047982A1 (en) | Image processing method and apparatus, electronic device and computer-readable storage medium | |
| CN107707835A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107707838A (en) | Image processing method and device | |
| CN107705278A (en) | The adding method and terminal device of dynamic effect | |
| CN107644440A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107610078A (en) | Image processing method and device | |
| CN107454336B (en) | Image processing method and apparatus, electronic apparatus, and computer-readable storage medium | |
| CN107734283A (en) | Picture processing method, device and the storage medium of Video chat |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong Applicant after: OPPO Guangdong Mobile Communications Co., Ltd. Address before: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong Applicant before: Guangdong OPPO Mobile Communications Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |