CN107454376A - Fall based reminding method, device, electronic installation and computer-readable recording medium - Google Patents
Fall based reminding method, device, electronic installation and computer-readable recording medium Download PDFInfo
- Publication number
- CN107454376A CN107454376A CN201710811958.2A CN201710811958A CN107454376A CN 107454376 A CN107454376 A CN 107454376A CN 201710811958 A CN201710811958 A CN 201710811958A CN 107454376 A CN107454376 A CN 107454376A
- Authority
- CN
- China
- Prior art keywords
- depth
- active user
- depth image
- image
- edge pixel
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G08—SIGNALLING
- G08B—SIGNALLING OR CALLING SYSTEMS; ORDER TELEGRAPHS; ALARM SYSTEMS
- G08B21/00—Alarms responsive to a single specified undesired or abnormal condition and not otherwise provided for
- G08B21/18—Status alarms
- G08B21/24—Reminder alarms, e.g. anti-loss alarms
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
- H04N13/106—Processing image signals
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N2013/0074—Stereoscopic image analysis
- H04N2013/0081—Depth or disparity estimation from stereoscopic image signals
Abstract
The invention discloses one kind to fall based reminding method, device and computer-readable recording medium.Wherein method includes：The first depth image of active user is obtained, and obtains the second depth image of active user place current scene；Edge extracting is carried out to the second depth image to obtain depth data of the edge pixel of each object in the second depth image in the second depth image；According to the depth data of the edge pixel of each object in the first depth image and the second depth image of active user, judge that the front of active user in current scene whether there is step；If so, then generate prompting message and be supplied to active user.The embodiment of the present invention can be gone out with accurate calibration plantar foot area sole edge pixel and object area edge pixel further, based on more accurately sole edge pixel and the depth data of object edge pixel carry out active user in the current scene front it is better with the presence or absence of the judgement of step, improve Consumer's Experience.
Description
Technical field
The present invention relates to technical field of image processing, more particularly to one kind to fall based reminding method, device, electronic installation and meter
Calculation machine readable storage medium storing program for executing.
Background technology
In correlation technique, when user's walking is had such as step in the virtual environment scene of difference in height in the background, lead to
Feature point extraction character contour and contour of object often are used, and current virtual scene is judged according to character contour and contour of object
The front of middle active user whether there is step.It is but this accurate using the character contour and contour of object of feature point extraction
Degree is not high, especially can not accurate calibration go out the border of personage and object, influence in the current virtual scene before active user
Side whether there is the judgement effect of step.
The content of the invention
The purpose of the present invention is intended at least solve one of above-mentioned technical problem to a certain extent.
Therefore, first purpose of the present invention is to propose that one kind falls based reminding method.This method can be gone out with accurate calibration
The sole edge pixel of plantar foot area and the edge pixel of object area further, based on more accurately sole edge pixel
Imitated with the presence or absence of the judgement of step in the front that active user in the current scene is carried out with the depth data of object edge pixel
Fruit more preferably, improves Consumer's Experience.
Second object of the present invention is to propose that one kind falls alarm set.
Third object of the present invention is to propose a kind of electronic installation.
Fourth object of the present invention is to propose a kind of computer-readable recording medium.
To reach above-mentioned purpose, what first aspect present invention embodiment proposed falls based reminding method, including：Obtain current use
First depth image at family, and obtain the second depth image of active user place current scene；To second depth
Image carries out edge extracting to obtain the edge pixel of each object in second depth image in second depth image
In depth data；According to the edge of each object in the first depth image of the active user and second depth image
The depth data of pixel, judge that the front of active user in the current scene whether there is step；If so, then letter is reminded in generation
Cease and be supplied to the active user.
To reach above-mentioned purpose, what second aspect of the present invention embodiment proposed falls alarm set, including：Depth image is adopted
Collect component, the depth image acquisition component is used for the first depth image for obtaining active user, and obtains the active user
Second depth image of place current scene；Processor, the processor are used for：Edge is carried out to second depth image to carry
Take to obtain depth data of the edge pixel of each object in second depth image in second depth image；Root
According to the depth data of the edge pixel of each object in the first depth image of the active user and second depth image,
Judge that the front of active user in the current scene whether there is step；If so, then generate prompting message and be supplied to described
Active user.
To reach above-mentioned purpose, electronic installation that third aspect present invention embodiment proposes, including：One or more processing
Device；Memory；With one or more programs, wherein one or more of programs are stored in the memory, and by
It is configured to by one or more of computing devices, described program includes being used to perform described in first aspect present invention embodiment
The instruction for falling based reminding method.
To reach above-mentioned purpose, computer-readable recording medium that fourth aspect present invention embodiment proposes, including with energy
The computer program that the electronic installation enough imaged is used in combination, the computer program can be executed by processor to complete the present invention
Fall based reminding method described in first aspect embodiment.
Embodiment of the present invention falls based reminding method, falls alarm set, electronic installation and computer-readable storage medium
Matter, by obtaining the depth data of sole edge pixel from the first depth image of active user, and work as where active user
The depth data of the object edge pixel of each object is obtained in second depth image of preceding scene, and according to sole edge pixel
To realize, whether active user is moved at step edge in the current scene with the depth data of object edge pixel.Due to
The acquisition of depth image, which is not easy the factor such as COLOR COMPOSITION THROUGH DISTRIBUTION in by illumination, scene, to be influenceed, and therefore, is extracted by depth image
People's plantar foot area and object area are more accurate, it is particularly possible to which accurate calibration goes out sole edge pixel and the object areas of plantar foot area
The edge pixel in domain.Further, based on more accurately sole edge pixel and the depth data of object edge pixel are carried out
The front of active user is better with the presence or absence of the judgement of step in the current scene, improves Consumer's Experience.
The additional aspect of the present invention and advantage will be set forth in part in the description, and will partly become from the following description
Obtain substantially, or recognized by the practice of the present invention.
Brief description of the drawings
In order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is the required accompanying drawing used in technology description to be briefly described, it should be apparent that, drawings in the following description are only this
Some embodiments of invention, for those of ordinary skill in the art, on the premise of not paying creative work, can be with
Other accompanying drawings are obtained according to these accompanying drawings.
Fig. 1 is the flow chart for falling based reminding method of some embodiments of the present invention；
Fig. 2 is the module diagram for falling alarm set of some embodiments of the present invention；
Fig. 3 is the structural representation of the electronic installation of some embodiments of the present invention；
Fig. 4 is the flow chart for falling based reminding method of some embodiments of the present invention；
Fig. 5 is the flow chart for falling based reminding method of some embodiments of the present invention；
Fig. 6 (a) to Fig. 6 (e) is the schematic diagram of a scenario of structural light measurement according to an embodiment of the invention；
Fig. 7 (a) and Fig. 7 (b) structural light measurements according to an embodiment of the invention schematic diagram of a scenario；
Fig. 8 is the flow chart for falling based reminding method of some embodiments of the present invention；
Fig. 9 is the module diagram of the electronic installation of some embodiments of the present invention；
Figure 10 is the module diagram of the electronic installation of some embodiments of the present invention.
Embodiment
In order to make the purpose , technical scheme and advantage of the present invention be clearer, it is right below in conjunction with drawings and Examples
The present invention is further elaborated.It should be appreciated that the specific embodiments described herein are merely illustrative of the present invention, and
It is not used in the restriction present invention.
It is appreciated that term " first " used in the present invention, " second " etc. can be used to describe various elements herein,
But these elements should not be limited by these terms.These terms are only used for distinguishing first element and another element.Citing comes
Say, without departing from the scope of the invention, the first client can be referred to as the second client, and similarly, can incite somebody to action
Second client is referred to as the first client.First client and the second client both clients, but it is not same visitor
Family end.
Below with reference to the accompanying drawings describe the embodiment of the present invention falls based reminding method, device, electronic installation and computer-readable
Storage medium.
Also referring to Fig. 1 to 2, the electronics for falling based reminding method and can be applied to the embodiment of the present invention of the embodiment of the present invention
Device 1000.This, which falls based reminding method, to include：
S110, obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image.
S120, edge extracting is carried out to the second depth image to obtain the edge pixel of each object in the second depth image
Depth data in the second depth image.
As a kind of example, edge extracting can be carried out to second depth image by Canny operators.Wherein, Canny
The core that operator carries out the algorithm of edge extracting mainly includes the following steps：First, with 2D gaussian filterings template to the second depth
Image carries out convolution to eliminate noise；Then, the Grad of the gray scale of each pixel is obtained using differential operator, and according to gradient
Value calculates the gradient direction of the gray scale of each pixel, and adjoining picture of the respective pixel along gradient direction can be found by gradient direction
Element；Then, each pixel is traveled through, if the gray scale of the gray value of some pixel and former and later two adjacent pixels on its gradient direction
It not is maximum that value, which is compared, then it is not marginal point to think this pixel.Locate in this way, can determine that in second depth image
In the pixel of marginal position, so as to obtain the edge pixel of each object after edge extracting, and then each object can be obtained
Depth data of the edge pixel in second depth image.
S130, according to the depth of the edge pixel of each object in the first depth image and the second depth image of active user
Degrees of data, judge that the front of active user in current scene whether there is step.
S140, if so, then generating prompting message and being supplied to active user.
Referring to Fig. 3, the embodiment of the present invention falls based reminding method and can fall alarm set by the embodiment of the present invention
100 realize.The electronic installation 1000 for falling alarm set 100 and can be used for the embodiment of the present invention of the embodiment of the present invention.Fall and carry
Device 100 of waking up may include depth image acquisition component 10 and processor 20.Above-mentioned steps S110 can gather group by depth image
Part 10 realizes that step S120~step S140 can be realized by processor 20.
That is, depth image acquisition component 10 can be used for the first depth image for obtaining active user, and obtain institute
Second depth image of current scene where stating active user；Processor 20 can be used for carrying out edge to second depth image
Extract to obtain depth data of the edge pixel of each object in second depth image in second depth image,
And according to the first depth image of the active user and the depth data of the edge pixel of each object, judge described work as
The front of active user whether there is step in preceding scene, if so, then generating prompting message and being supplied to the active user.
Wherein, second depth image characterizes each personal or object depth information in the background for including active user,
The first depth image of the active user characterizes the depth information of personage in the background comprising the active user.Described second
Each pixel in depth image can be found in first depth image to should pixel depth information.
The electronic installation 1000 for falling alarm set 100 and can apply to the embodiment of the present invention of the embodiment of the present invention.
That is the electronic installation 1000 of the embodiment of the present invention may include the embodiment of the present invention fall alarm set 100.
In some embodiments, the electronic installation 1000 can have shoot function, and the shoot function is to utilize knot
Structure light principle carries out the shooting of depth image.For example, the electronic installation 1000 can be smart mobile phone, platform computer, intelligent head
Helmet, intelligent glasses etc.；It can also be VR (Virtual Reality, virtual reality technology) equipment, AR (Augmented
Reality, augmented reality) equipment etc..
Influenceed because the acquisition of depth image is not easy the factor such as COLOR COMPOSITION THROUGH DISTRIBUTION in by illumination, scene, and in depth image
Comprising depth information there is higher accuracy, therefore, the personage edge that is extracted by the first depth image and logical
Cross the object edge that the second depth image extracts, more accurately, it is particularly possible to accurate calibration go out personage's sole edge pixel and
Object edge pixel.Further, based on more accurately personage's sole edge pixel and the depth data of object edge pixel
The front for carrying out active user in current scene is better with the presence or absence of the judgement of step.
As a kind of example, referring to Fig. 4, in some embodiments, above-mentioned steps S110 obtains the of active user
The step of one depth image, may include：
S1101, to active user's projective structure light；
S1102, shoot the structure light image modulated through active user；
S1103, phase information corresponding to each pixel of demodulation structure light image is to obtain the first depth image.
Referring again to Fig. 3, in some embodiments, depth image acquisition component 10 may include the He of structured light projector 11
Structure light video camera head 12.Step S1101 can be realized that step S1102 and step S1103 can be by tying by structured light projector 11
Structure light video camera head 12 is realized.
That is, structured light projector 11 can be used for active user's projective structure light；Structure light video camera head 12 can be used
In the structure light image that shooting is modulated through the active user, and demodulate phase corresponding to each pixel of the structure light image
Information is to obtain first depth image.
For example, structured light projector 11 can be by the face and body of the project structured light of certain pattern to active user
After upper, the structure light image after being modulated by active user can be formed in the face of active user and the surface of body.Structure light is taken the photograph
Structure light image after the shooting of picture first 12 is modulated, then structure light image is demodulated deep to obtain the first of the active user
Spend image.Wherein, the pattern of structure light can be laser stripe, Gray code, sine streak, non-homogeneous speckle etc..
Referring to Fig. 5, in some embodiments, phase corresponding to each pixel of step S1103 demodulation structure light images
The step of position information is to obtain the first depth image may include：
S11031, phase information corresponding to each pixel in demodulation structure light image；
S11032, phase information is converted into depth information；
S11033, the first depth image of the active user is generated according to depth information.
Referring again to Fig. 2, in some embodiments, step S11031, step S11032 and step S11033 can be with
Realized by structure light video camera head 12.
That is, structure light video camera head 12 can be further used in demodulation structure light image phase corresponding to each pixel
Information, the phase information is converted into depth information, and first depth image is generated according to the depth information.
For example, compared with non-modulated structure light, the phase information of the structure light after modulation is changed,
The structure light showed in structure light image is to generate the structure light after distortion, wherein, the phase information of change can table
Levy the depth information of object.Therefore, structure light video camera head 12 demodulates phase corresponding to each pixel in structure light image first
Information, depth information is calculated further according to phase information, so as to obtain the first final depth image.
In order that those skilled in the art is more apparent from gathering the face of active user and body according to structure
The process of first depth image of body, explained below by taking a kind of widely used optical grating projection technology (fringe projection technology) as an example
State its concrete principle.Wherein, optical grating projection technology belongs to sensu lato area-structure light.
As shown in Fig. 6 (a), when being projected using area-structure light, sine streak is produced by computer programming first,
And sine streak is projected to measured object by structured light projector 11, recycle structure light video camera head 12 to shoot striped by object
Degree of crook after modulation, then demodulate the curved stripes and obtain phase, then phase is converted into depth information to obtain
One depth image.The problem of to avoid producing error or error coupler, needed before carrying out depth information collection using structure light to depth
Spend image collection assembly 10 and carry out parameter calibration, demarcation includes geometric parameter (for example, structure light video camera head 12 and project structured light
Relative position parameter between device 11 etc.) demarcation, the inner parameter and structured light projector 11 of structure light video camera head 12
Demarcation of inner parameter etc..
Specifically, the first step, computer programming produce sine streak.Need to obtain using the striped of distortion due to follow-up
Phase, for example phase is obtained using four step phase-shifting methods, therefore produce four width phase differences here and beStriped, then structure light throw
Emitter 11 projects the four spokes line timesharing on measured object (mask as shown in Fig. 6 (a)), and structure light video camera head 12 collects
Such as the figure on Fig. 6 (b) left sides, while to read the striped of the plane of reference shown on the right of Fig. 6 (b).
Second step, carry out phase recovery.The bar graph that structure light video camera head 12 is modulated according to four width collected (is tied
Structure light image) to calculate the phase diagram by phase modulation, now obtained be to block phase diagram.Because what four step Phase-shifting algorithms obtained
Result is to calculate gained by arctan function, therefore the phase after structure light modulation is limited between [- π, π], that is to say, that
Phase after modulation exceedes [- π, π], and it can restart again.Shown in the phase main value such as Fig. 6 (c) finally given.
Wherein, it is necessary to carry out the saltus step processing that disappears, it is continuous phase that will block phase recovery during phase recovery is carried out
Position.As shown in Fig. 6 (d), the left side is the continuous phase bitmap modulated, and the right is to refer to continuous phase bitmap.
3rd step, subtract each other to obtain phase difference (i.e. phase information) by the continuous phase modulated and with reference to continuous phase, should
Phase difference characterizes depth information of the measured object with respect to the plane of reference, then phase difference is substituted into conversion formula (its of phase and depth
In, the parameter being related in formula is by demarcation), you can obtain the threedimensional model of the object under test as shown in Fig. 6 (e).
It should be appreciated that in actual applications, according to the difference of concrete application scene, employed in the embodiment of the present invention
Structure light in addition to above-mentioned grating, can also be other arbitrary graphic patterns.
As a kind of possible implementation, the depth information of pattern light progress active user also can be used in the present invention
Collection.
Specifically, the method that pattern light obtains depth information is that this spreads out using a diffraction element for being essentially flat board
The relief diffraction structure that there are element particular phases to be distributed is penetrated, cross section is with two or more concavo-convex step embossment knots
Structure.Substantially 1 micron of the thickness of substrate in diffraction element, each step it is highly non-uniform, the span of height can be 0.7
Micron~0.9 micron.Structure shown in Fig. 7 (a) is the local diffraction structure of the collimation beam splitting element of the present embodiment.Fig. 7 (b) is edge
The unit of the cross sectional side view of section A-A, abscissa and ordinate is micron.The speckle pattern of pattern photogenerated has
The randomness of height, and can with the difference of distance changing patterns.Therefore, depth information is being obtained using pattern light
Before, it is necessary first to the speckle pattern in space is calibrated, for example, in the range of 0~4 meter of distance structure light video camera head 12,
A reference planes are taken every 1 centimetre, then just save 400 width speckle images after demarcating, the spacing of demarcation is smaller, obtains
Depth information precision it is higher.Then, structured light projector 11 is by pattern light projection to measured object (such as active user)
On, the speckle pattern that the difference in height on measured object surface to project the pattern light on measured object changes.Structure light
After camera 12 shoots the speckle pattern (i.e. structure light image) projected on measured object, then by after speckle pattern and demarcation early stage
The 400 width speckle images preserved carry out computing cross-correlation one by one, and then obtain 400 width correlation chart pictures.Testee in space
The position at place can show peak value on correlation chart picture, above-mentioned peak value is superimposed and after interpolation arithmetic
Obtain the depth information of measured object.
Most diffraction lights are obtained after diffraction is carried out to light beam due to common diffraction element, but per beam diffraction light light intensity difference
Greatly, it is also big to the risk of human eye injury.Re-diffraction even is carried out to diffraction light, the uniformity of obtained light beam is relatively low.
Therefore, the effect projected using the light beam of common diffraction element diffraction to measured object is poor.Using collimation in the present embodiment
Beam splitting element, the element not only have the function that to collimate uncollimated rays, also have the function that light splitting, i.e., through speculum
The non-collimated light of reflection is emitted multi-beam collimation light beam, and the multi-beam collimation being emitted after collimating beam splitting element toward different angles
The area of section approximately equal of light beam, flux of energy approximately equal, and then to carry out using the scatterplot light after the beam diffraction
The effect of projection is more preferable.Meanwhile laser emitting light is dispersed to every light beam, the risk of injury human eye is reduce further, and dissipate
Spot structure light is for other uniform structure lights of arrangement, when reaching same collection effect, the consumption of pattern light
Electricity is lower.
It should also be noted that, the implementation of the first depth image of above-mentioned acquisition active user, is also applied for above-mentioned
The acquisition of second depth image of current scene, the descriptions of the acquisition modes of its second depth image can be found in above-mentioned current
The description of the acquisition modes of the first depth image of user, will not be repeated here.Referring to Fig. 8, in some embodiments, step
Rapid S130 according to the depth data of the edge pixel of each object in the first depth image and the second depth image of active user,
Judge that the step of front of active user in current scene whether there is step may include：
S1301, personage's edge extracting is carried out to determine that the first of active user is deep to the first depth image of active user
Spend the pixel of the marginal position object of personage's sole in image；
S1302, obtain depth data of the sole edge pixel in the first depth image；
S1303, according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data, judge that the front of active user in current scene whether there is step.
, can be according to the side of each object in the depth data and the second depth image of sole edge pixel as a kind of example
The depth data of edge pixel, judge the target object edge pixel in sole edge pixel and the second depth image whether in level
Difference in height on direction be present, step in the current scene in front of active user be present if so, then can determine that, otherwise judge institute
Step is not present in the front for stating active user in current scene.
Referring again to Fig. 2, in some embodiments, step S1301, step S1302 and step S1303 can be by
Reason device 20 is realized.
That is, processor 20 can be further used for carrying out personage's edge extracting to the first depth image of active user
To determine pixel corresponding to the marginal position of personage in the first depth image of active user, and obtain sole edge pixel and working as
Depth data in the first depth image of preceding user, and according to object edge in sole edge pixel and the current scene
The depth data of pixel judges that the front of active user in the current scene whether there is step, wherein, when sole edge picture
When element has difference in height in the horizontal direction with the target object edge pixel in the second depth image, the current field can determine that
Step in scape in front of active user be present.
As a kind of example, processor 20 can be carried out by Canny operators to the first depth image of the active user
Edge extracting.Wherein, the core of the algorithm of Canny operators progress edge extracting mainly includes the following steps：First, with 2D Gausses
Filtering Template carries out convolution to eliminate noise to the first depth image of active user；Then, obtained using differential operator each
The Grad of the gray scale of pixel, and calculate according to Grad the gradient direction of the gray scale of each pixel, can be with by gradient direction
Find adjacent pixels of the respective pixel along gradient direction；Then, each pixel is traveled through, if the gray value of some pixel and its ladder
It is not maximum that the gray value of former and later two adjacent pixels, which is compared, on degree direction, then it is not marginal point to think this pixel.Such as
This, you can determine to be in the pixel of marginal position in the first depth image of the active user, so as to obtain edge extracting
The edge pixel of personage afterwards, and then depth data of the edge pixel of personage in first depth image can be obtained.
Processor 20, can be according to described when obtaining depth data of the sole edge pixel in first depth image
Sole edge pixel and the depth data of object edge pixel, to judge platform being present in front of active user in the current scene
Rank.For example, active user walks when with step etc. with the environment scene of difference in height, due to the pixel with difference in height
It is discontinuous in the horizontal direction, it is possible to using the characteristic, to judge whether active user marches to step edge, if
The active user is marched at step edge, then the active user can be reminded.
Processor 20 can generate prompting message simultaneously when step be present in front of active user in judging the current scene
It is supplied to active user.As a kind of example, the prompting message can be supplied to by one or more of mode described
Active user：Background side in mode, the mode of vibrating alert and change display device that the mode of voice broadcast, text show
Mode of edge color etc..
That is, processor 20 when generating prompting message, can be believed the prompting by one or more of mode
Breath is supplied to active user, to remind active user's " front has step, please be careful ", for example, can be shaken by mobile phone
Mode, which is reminded in front of user, has step, or, reminded by way of speech play in front of user and step be present, or, lead to
Cross the mode that text shows to show the text in a display device, step be present in front of user to remind, or, pass through change
In display device there is step in the mode of background edge color to remind in front of user.
Also referring to Fig. 3 and Fig. 9, the embodiment of the present invention also proposed a kind of electronic installation 1000.Electronic installation 1000
It may include to fall alarm set 100.Falling alarm set 100 can utilize hardware and/or software to realize.Fall alarm set
100 may include depth image acquisition component 10 and processor 20.
Specifically, depth image acquisition component 10 may include structured light projector 11 and structure light video camera head 12, depth map
The depth information that can be used for catching active user as acquisition component 10 and is caught with obtaining the first depth image of the active user
The depth information of current scene where catching the active user is to obtain the second depth image.For example, with depth image collection group
Part 10 catches the depth information of active user exemplified by obtaining the first depth image, and structured light projector 11 can be used for structure light
Active user is projected to, wherein, structured light patterns can be dissipating for laser stripe, Gray code, sine streak or random alignment
Spot pattern etc..Structure light video camera head 12 includes imaging sensor 121 and lens 122, and the number of lens 122 can be one or more
It is individual.Imaging sensor 121 can be used for capturing structure light projector 11 to be projected to the structure light image on active user.Structure light figure
As can be sent by depth image acquisition component 10 to processor 20 be demodulated, the processing such as phase recovery, phase information calculate with
Obtain the depth information of active user.It is appreciated that the embodiment of the depth information of the current scene can be found in above-mentioned work as
The embodiment of the depth information of preceding user, will not be repeated here.
In some embodiments, falling alarm set 100 may include imaging device 110, and the imaging device 110 may include
Depth image acquisition component 10 and visible image capturing first 111.Wherein, the visible image capturing first 111 can be used for catching made thing body
Colour information to obtain coloured image.The visible image capturing first 111 and the function of structure light video camera head 12 can be by a shootings
Head is realized, that is to say, that imaging device 10 only includes a camera and a structured light projector 11, and above-mentioned camera is not only
Structure light image can also be shot with shoot coloured image.
Except using structure light obtain depth image in addition to, can also by binocular vision method, based on differential time of flight (Time
Of Flight, TOF) to obtain the second of the first depth image of active user and current scene as acquisition methods deep for even depth
Spend image.
Processor 20 further carries out edge extracting to obtain in second depth image to second depth image
Depth data of the edge pixel of each object in second depth image, and to the first depth map of the active user
As carrying out personage's edge extracting with pixel corresponding to determining the marginal position of personage in first depth image, and obtain sole
Depth data of the edge pixel in first depth image, when detecting the sole edge pixel and target object edge
It during pixel target object edge pixel, can determine that step in the current scene in front of active user be present, now, can generate
Prompting message is simultaneously supplied to the active user.
In addition, falling alarm set 100 also includes video memory 30.Video memory 30 can be embedded in electronic installation
In 1000 or independently of the memory outside electronic installation 1000, and it may include direct memory access (DMA) (Direct
Memory Access, English abbreviation DMA) feature.The raw image data or depth image of first 111 collection of visible image capturing
The structure light image related data that acquisition component 10 gathers, which can transmit, to be stored or is cached into video memory 30.Processing
Device 20 can read structure light image related data to be handled to obtain the first of the active user from video memory 30
Second depth image of depth image and the current scene.In addition, the first depth image of the active user and it is described work as
Second depth image of preceding scene is also storable in video memory 30, and calling is handled device 20 for processing at any time, for example, place
Reason device 20 calls the first depth image of the active user to carry out personage's edge extracting, and calls second depth image to enter
The edge extracting of each object in row scene.Wherein, can be by personage's sole edge pixel of acquisition and sole edge pixel
Depth data is stored in video memory 30, can also be by the object edge pixel in second depth of acquisition and described
The depth data of object edge pixel is stored in video memory 30.
Fall alarm set 100 and may also include display 50.Display 50 can obtain prompting letter directly from processor 20
Breath.Display 50 shows that the prompting message has step to remind in the current scene in front of active user, please be careful.Fall
Fall alarm set 100 and may also include encoder/decoder 60, encoder/decoder 60 can active user described in encoding and decoding
The view data of one depth image and the second depth image of the current scene.The view data of coding can be stored in figure
In picture memory 30, and it can be shown before image is shown on display 50 by decoder decompresses.Encoder/
Decoder 60 can be realized by central processing unit (Central Processing Unit, CPU), GPU or coprocessor.In other words,
Encoder/decoder 60 can be in central processing unit (Central Processing Unit, CPU), GPU and coprocessor
Any one or more.
Falling alarm set 100 also includes control logic device 40.Imaging device 10 imaging when, processor 20 can according into
As the data that equipment obtains are analyzed to determine one or more control parameters of imaging device 10 (for example, time for exposure etc.)
Image statistics.Processor 20 sends image statistics to control logic device 40, the control imaging of control logic device 40
Equipment 10 is imaged with the control parameter determined.Control logic device 40 may include to perform one or more routines (such as firmware)
Processor and/or microcontroller.One or more routines can determine imaging device 10 according to the image statistics of reception
Control parameter.
Referring to Fig. 10, the electronic installation 1000 of the embodiment of the present invention may include one or more processors 200, memory
300 and one or more programs 310.Wherein one or more programs 310 are stored in memory 300, and are configured to
Performed by one or more processors 200.What program 310 included being used for performing above-mentioned any one embodiment falls prompting side
The instruction of method.
For example, program 310 may include for performing the instruction for falling based reminding method described in following steps：
S110 ', obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image；
S120 ', edge extracting is carried out to the second depth image to obtain the edge picture of each object in the second depth image
Depth data of the element in the second depth image；
S130 ', according to the edge pixel of each object in the first depth image and the second depth image of active user
Depth data, judge that the front of active user in current scene whether there is step；
S140 ', if so, then generating prompting message and being supplied to active user.
For another example program 310 also includes being used to perform the instruction for falling based reminding method described in following steps：
S11031 ', phase information corresponding to each pixel in demodulation structure light image；
S11032 ', phase information is converted into depth information；
S11033 ', the first depth image is generated according to depth information.
In another example program 310 also includes being used to perform the instruction for falling based reminding method described in following steps：
S1301 ', personage's edge extracting is carried out to determine that the first of active user is deep to the first depth image of active user
Spend the pixel of the marginal position object of personage's sole in image；
S1302 ', obtain depth data of the sole edge pixel in the first depth image；
S1303 ', according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data, judge that the front of active user in current scene whether there is step.
As a kind of example, according to the edge of each object in the depth data of sole edge pixel and the second depth image
The depth data of pixel, judge the target object edge pixel in sole edge pixel and the second depth image whether in level side
Difference in height be present upwards, if so, then judging step be present in front of active user in current scene.
The computer-readable recording medium of embodiment of the present invention includes being combined with the electronic installation 1000 that can be imaged making
Computer program.Computer program can be performed by processor 200 to be carried with completing above-mentioned falling for any one embodiment
Awake method.
For example, computer program can be performed by processor 200 to complete to fall based reminding method described in following steps：
S110 ', obtains the first depth image of active user, and obtains the second depth of active user place current scene
Image；
S120 ', edge extracting is carried out to the second depth image to obtain the edge picture of each object in the second depth image
Depth data of the element in the second depth image；
S130 ', according to the edge pixel of each object in the first depth image and the second depth image of active user
Depth data, judge that the front of active user in current scene whether there is step；
S140 ', if so, then generating prompting message and being supplied to active user.
For another example computer program can be performed by processor 200 to complete to fall based reminding method described in following steps：
S11031 ', phase information corresponding to each pixel in demodulation structure light image；
S11032 ', phase information is converted into depth information；
S11033 ', the first depth image is generated according to depth information.
In another example computer program can be performed by processor 200 to complete to fall based reminding method described in following steps：
S1301 ', personage's edge extracting is carried out to determine that the first of active user is deep to the first depth image of active user
Spend the pixel of the marginal position object of personage's sole in image；
S1302 ', obtain depth data of the sole edge pixel in the first depth image；
S1303 ', according to the edge pixel of each object in the depth data of sole edge pixel and the second depth image
Depth data, judge that the front of active user in current scene whether there is step.
As a kind of example, according to the edge of each object in the depth data of sole edge pixel and the second depth image
The depth data of pixel, judge the target object edge pixel in sole edge pixel and the second depth image whether in level side
Difference in height be present upwards, if so, then judging step be present in front of active user in current scene.
To sum up, embodiment of the present invention falls based reminding method, alarm set, electronic installation and computer-readable storage medium
Matter, by obtaining the depth data of sole edge pixel from the first depth image of active user, and work as where active user
The depth data of the object edge pixel of each object is obtained in second depth image of preceding scene, and according to sole edge pixel
To realize, whether active user is moved at step edge in the current scene with the depth data of object edge pixel.Due to
The acquisition of depth image, which is not easy the factor such as COLOR COMPOSITION THROUGH DISTRIBUTION in by illumination, scene, to be influenceed, and therefore, is extracted by depth image
People's plantar foot area and object area are more accurate, it is particularly possible to which accurate calibration goes out sole edge pixel and the object areas of plantar foot area
The edge pixel in domain.Further, based on more accurately sole edge pixel and the depth data of object edge pixel are carried out
The front of active user is better with the presence or absence of the judgement of step in the current scene, improves Consumer's Experience.
In the description of this specification, reference term " one embodiment ", " some embodiments ", " example ", " specifically show
The description of example " or " some examples " etc. means specific features, structure, material or the spy for combining the embodiment or example description
Point is contained at least one embodiment or example of the present invention.In this manual, to the schematic representation of above-mentioned term not
Identical embodiment or example must be directed to.Moreover, specific features, structure, material or the feature of description can be with office
Combined in an appropriate manner in one or more embodiments or example.In addition, in the case of not conflicting, the skill of this area
Art personnel can be tied the different embodiments or example and the feature of different embodiments or example described in this specification
Close and combine.
In addition, term " first ", " second " are only used for describing purpose, and it is not intended that instruction or hint relative importance
Or the implicit quantity for indicating indicated technical characteristic.Thus, define " first ", the feature of " second " can be expressed or
Implicitly include at least one this feature.In the description of the invention, " multiple " are meant that at least two, such as two, three
It is individual etc., unless otherwise specifically defined.
Any process or method described otherwise above description in flow chart or herein is construed as, and represents to include
Module, fragment or the portion of the code of the executable instruction of one or more the step of being used to realize specific logical function or process
Point, and the scope of the preferred embodiment of the present invention includes other realization, wherein can not press shown or discuss suitable
Sequence, including according to involved function by it is basic simultaneously in the way of or in the opposite order, carry out perform function, this should be of the invention
Embodiment person of ordinary skill in the field understood.
Expression or logic and/or step described otherwise above herein in flow charts, for example, being considered use
In the order list for the executable instruction for realizing logic function, may be embodied in any computer-readable medium, for
Instruction execution system, device or equipment (such as computer based system including the system of processor or other can be held from instruction
The system of row system, device or equipment instruction fetch and execute instruction) use, or combine these instruction execution systems, device or set
It is standby and use.For the purpose of this specification, " computer-readable medium " can any can be included, store, communicate, propagate or pass
Defeated program is for instruction execution system, device or equipment or the dress used with reference to these instruction execution systems, device or equipment
Put.The more specifically example (non-exhaustive list) of computer-readable medium includes following：Electricity with one or more wiring
Connecting portion (electronic installation), portable computer diskette box (magnetic device), random access memory (RAM), read-only storage
(ROM), erasable edit read-only storage (EPROM or flash memory), fiber device, and portable optic disk is read-only deposits
Reservoir (CDROM).In addition, computer-readable medium, which can even is that, to print the paper of described program thereon or other are suitable
Medium, because can then enter edlin, interpretation or if necessary with it for example by carrying out optical scanner to paper or other media
His suitable method is handled electronically to obtain described program, is then stored in computer storage.
It should be appreciated that each several part of the present invention can be realized with hardware, software, firmware or combinations thereof.Above-mentioned
In embodiment, software that multiple steps or method can be performed in memory and by suitable instruction execution system with storage
Or firmware is realized.If, and in another embodiment, can be with well known in the art for example, realized with hardware
Any one of row technology or their combination are realized：With the logic gates for realizing logic function to data-signal
Discrete logic, have suitable combinational logic gate circuit application specific integrated circuit, programmable gate array (PGA), scene
Programmable gate array (FPGA) etc..
Those skilled in the art are appreciated that to realize all or part of step that above-described embodiment method carries
Suddenly it is that by program the hardware of correlation can be instructed to complete, described program can be stored in a kind of computer-readable storage medium
In matter, the program upon execution, including one or a combination set of the step of embodiment of the method.
In addition, each functional unit in each embodiment of the present invention can be integrated in a processing module, can also
That unit is individually physically present, can also two or more units be integrated in a module.Above-mentioned integrated mould
Block can both be realized in the form of hardware, can also be realized in the form of software function module.The integrated module is such as
Fruit is realized in the form of software function module and as independent production marketing or in use, can also be stored in a computer
In read/write memory medium.
Storage medium mentioned above can be read-only storage, disk or CD etc..Although have been shown and retouch above
Embodiments of the invention are stated, it is to be understood that above-described embodiment is exemplary, it is impossible to be interpreted as the limit to the present invention
System, one of ordinary skill in the art can be changed to above-described embodiment, change, replace and become within the scope of the invention
Type.
Claims (14)
1. one kind falls based reminding method, it is characterised in that comprises the following steps：
The first depth image of active user is obtained, and obtains the second depth image of active user place current scene；
Edge extracting is carried out to second depth image to obtain the edge pixel of each object in second depth image
Depth data in second depth image；
According to the depth of the edge pixel of each object in the first depth image of the active user and second depth image
Degrees of data, judge that the front of active user in the current scene whether there is step；
If so, then generate prompting message and be supplied to the active user.
2. fall based reminding method as claimed in claim 1, it is characterised in that first depth map for obtaining active user
Picture, including：
To active user's projective structure light；
The structure light image that shooting is modulated through the active user；
Phase information corresponding to each pixel of the structure light image is demodulated to obtain first depth image.
3. fall based reminding method as claimed in claim 2, it is characterised in that each picture of the demodulation structure light image
Phase information corresponding to element to obtain first depth image, including：
Demodulate phase information corresponding to each pixel in the structure light image；
The phase information is converted into depth information；
First depth image is generated according to the depth information.
4. fall based reminding method as claimed in claim 1, it is characterised in that first depth according to the active user
The depth data of the edge pixel of each object, judges currently to use in the current scene in image and second depth image
The front at family whether there is step, including：
Personage's edge extracting is carried out to determine the first depth of the active user to the first depth image of the active user
The pixel of the marginal position object of personage's sole in image；
Obtain depth data of the sole edge pixel in first depth image；
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degrees of data, judge that the front of active user in the current scene whether there is step.
5. fall based reminding method as claimed in claim 4, it is characterised in that the depth according to the sole edge pixel
The depth data of the edge pixel of each object, judges currently to use in the current scene in data and second depth image
The front at family whether there is step, including：
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degrees of data, judge the sole edge pixel with the target object edge pixel in second depth image whether in level side
Difference in height be present upwards；
If so, then judge step be present in front of active user in the current scene.
6. fall based reminding method as any one of claim 1 to 5, it is characterised in that pass through one or more of
The prompting message is supplied to the active user by mode：
Background edge color in mode, the mode of vibrating alert and change display device that the mode of voice broadcast, text show
Mode.
7. one kind falls alarm set, it is characterised in that including：
Depth image acquisition component, the depth image acquisition component is used for the first depth image for obtaining active user, and obtains
Second depth image of current scene where taking the active user；
Processor, the processor are used for：
Edge extracting is carried out to second depth image to obtain the edge pixel of each object in second depth image
Depth data in second depth image；
According to the depth of the edge pixel of each object in the first depth image of the active user and second depth image
Degrees of data, judge that the front of active user in the current scene whether there is step；
If so, then generate prompting message and be supplied to the active user.
8. fall alarm set as claimed in claim 7, it is characterised in that the depth image acquisition component includes structure light
The projector and structure light video camera head, the structured light projector are used for active user's projective structure light；
The structure light video camera head is used for：
The structure light image that shooting is modulated through the active user；
Phase information corresponding to each pixel of the structure light image is demodulated to obtain first depth image.
9. fall alarm set as claimed in claim 8, it is characterised in that the structure light video camera head is specifically used for：
Demodulate phase information corresponding to each pixel in the structure light image；
The phase information is converted into depth information；
First depth image is generated according to the depth information.
10. fall alarm set as claimed in claim 7, it is characterised in that the processor is specifically used for：
Personage's edge extracting is carried out to determine the first depth of the active user to the first depth image of the active user
The pixel of the marginal position object of personage's sole in image；
Obtain depth data of the sole edge pixel in first depth image；
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degrees of data, judge that the front of active user in the current scene whether there is step.
11. fall alarm set as claimed in claim 10, it is characterised in that the processor is specifically used for：
According to the depth of the edge pixel of each object in the depth data of the sole edge pixel and second depth image
Degrees of data, judge the sole edge pixel with the target object edge pixel in second depth image whether in level side
Difference in height be present upwards；
If so, then judge step be present in front of active user in the current scene.
12. fall alarm set as any one of claim 7 to 11, it is characterised in that by following a kind of or more
The prompting message is supplied to the active user by kind mode：
Background edge color in mode, the mode of vibrating alert and change display device that the mode of voice broadcast, text show
Mode.
13. a kind of electronic installation, it is characterised in that the electronic installation includes：
One or more processors；
Memory；With
One or more programs, wherein one or more of programs are stored in the memory, and be configured to by
One or more of computing devices, described program include carrying for falling any one of perform claim requirement 1 to 6
The instruction for method of waking up.
A kind of 14. computer-readable recording medium, it is characterised in that the meter being used in combination including the electronic installation with that can image
Calculation machine program, the computer program can be executed by processor to complete to fall prompting any one of claim 1 to 6
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710811958.2A CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710811958.2A CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107454376A true CN107454376A (en) | 2017-12-08 |
| CN107454376B CN107454376B (en) | 2019-07-19 |
Family
ID=60495300
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710811958.2A Active CN107454376B (en) | 2017-09-11 | 2017-09-11 | Fall based reminding method, device, electronic device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107454376B (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110266891A (en) * | 2019-06-28 | 2019-09-20 | 无锡睿勤科技有限公司 | Information cuing method, device, terminal and storage medium |
| CN111899348A (en) * | 2020-07-14 | 2020-11-06 | 四川深瑞视科技有限公司 | Projection-based augmented reality experiment demonstration system and method |
| CN112535476A (en) * | 2020-12-01 | 2021-03-23 | 业成科技（成都）有限公司 | Fall detection system and method thereof |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150016777A1 (en) * | 2012-06-11 | 2015-01-15 | Magic Leap, Inc. | Planar waveguide apparatus with diffraction element(s) and system employing same |
| CN104680525A (en) * | 2015-02-12 | 2015-06-03 | 南通大学 | Automatic human body fall-down detection method based on Kinect depth image |
| CN105934227A (en) * | 2014-01-24 | 2016-09-07 | 微软技术许可有限责任公司 | Audio navigation assistance |
| CN106126042A (en) * | 2016-07-01 | 2016-11-16 | 京东方科技集团股份有限公司 | A kind of environment based reminding method and system |
-
2017
- 2017-09-11 CN CN201710811958.2A patent/CN107454376B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150016777A1 (en) * | 2012-06-11 | 2015-01-15 | Magic Leap, Inc. | Planar waveguide apparatus with diffraction element(s) and system employing same |
| CN105934227A (en) * | 2014-01-24 | 2016-09-07 | 微软技术许可有限责任公司 | Audio navigation assistance |
| CN104680525A (en) * | 2015-02-12 | 2015-06-03 | 南通大学 | Automatic human body fall-down detection method based on Kinect depth image |
| CN106126042A (en) * | 2016-07-01 | 2016-11-16 | 京东方科技集团股份有限公司 | A kind of environment based reminding method and system |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110266891A (en) * | 2019-06-28 | 2019-09-20 | 无锡睿勤科技有限公司 | Information cuing method, device, terminal and storage medium |
| CN111899348A (en) * | 2020-07-14 | 2020-11-06 | 四川深瑞视科技有限公司 | Projection-based augmented reality experiment demonstration system and method |
| CN112535476A (en) * | 2020-12-01 | 2021-03-23 | 业成科技（成都）有限公司 | Fall detection system and method thereof |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107454376B (en) | 2019-07-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107610077A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107742296A (en) | Dynamic image generation method and electronic installation | |
| CN107797664A (en) | Content display method, device and electronic installation | |
| CN107707839A (en) | Image processing method and device | |
| CN107509045A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107734267A (en) | Image processing method and device | |
| CN107610127A (en) | Image processing method, device, electronic installation and computer-readable recording medium | |
| CN107742300A (en) | Image processing method, device, electronic installation and computer-readable recording medium | |
| CN107807806A (en) | Display parameters method of adjustment, device and electronic installation | |
| CN107707831A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107707838A (en) | Image processing method and device | |
| CN107507269A (en) | Personalized three-dimensional model generating method, device and terminal device | |
| CN107707835A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107705278A (en) | The adding method and terminal device of dynamic effect | |
| CN107509043A (en) | Image processing method and device | |
| CN107610078A (en) | Image processing method and device | |
| CN107734264A (en) | Image processing method and device | |
| CN107644440A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107734283A (en) | Picture processing method, device and the storage medium of Video chat | |
| CN107454376B (en) | Fall based reminding method, device, electronic device and computer readable storage medium | |
| CN107613228A (en) | The adding method and terminal device of virtual dress ornament | |
| CN107610076A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107454336A (en) | Image processing method and device, electronic installation and computer-readable recording medium | |
| CN107592491A (en) | Video communication background display methods and device | |
| CN107705243A (en) | Image processing method and device, electronic installation and computer-readable recording medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong Applicant after: OPPO Guangdong Mobile Communications Co., Ltd. Address before: 523860 No. 18, Wu Sha Beach Road, Changan Town, Dongguan, Guangdong Applicant before: Guangdong OPPO Mobile Communications Co., Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |