CN108564613A - A kind of depth data acquisition methods and mobile terminal - Google Patents
A kind of depth data acquisition methods and mobile terminal Download PDFInfo
- Publication number
- CN108564613A CN108564613A CN201810325685.5A CN201810325685A CN108564613A CN 108564613 A CN108564613 A CN 108564613A CN 201810325685 A CN201810325685 A CN 201810325685A CN 108564613 A CN108564613 A CN 108564613A
- Authority
- CN
- China
- Prior art keywords
- data
- rgb
- cameras
- depth
- mobile terminal
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
A kind of depth data acquisition methods of present invention offer and mobile terminal, wherein method include：Obtain the RGB data of the first RGB camera the image collected of the mobile terminal and the IR data of first infrared IR cameras the image collected；Using the RGB data and the IR data as the input data of target algorithm, corresponding depth data is calculated by the target algorithm；Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains to multigroup training data, and every group of training data includes the first RGB data, the first IR data and the first depth data under same scene.Corresponding depth data can be calculated according to RGB data, IR data and target algorithm in the embodiment of the present invention, so that the depth data that the embodiment of the present invention is calculated is closer to actual depth data, the accuracy of the depth data obtained by the imaging scheme of RGB camera combination IR cameras can be effectively improved.
Description
Technical field
The present invention relates to technical field of image processing more particularly to a kind of depth data acquisition methods and mobile terminal.
Background technology
With the continuous development of electronic information technology, the function of mobile terminal (such as smart mobile phone, tablet computer etc.) is got over
Come more powerful, 3D (Three Dimensions, three-dimensional) imaging technique becomes a kind of trend in image processing techniques, current
In 3D imaging technique, depth camera is indispensable.However, in the prior art, depth camera depends on processing chip
Structure design and computing capability, hardware cost is generally higher.The depth camera of binocular imaging uses two RGB cameras
It is carried out at the same time shooting, is based on principle of parallax, by forming the difference of two RGB images, then calculates the depth of subject
Degree information is not suitable for dim environment although hardware cost is low compared with other depth cameras.
So, the equally RGB camera combination IR cameras (Infrared based on principle of parallax has been derived
Radiation Camera, infrared camera) depth imaging scheme, that is, use RGB cameras and IR cameras to substitute and pass
Double RGB cameras of system carry out ranging.Wherein, infrared light is received by IR cameras and forms IR images, can solved traditional double
The problem of depth camera of mesh imaging is limited to dim environment.However, the program to the registration of RGB image and IR images have compared with
High requirement, the accuracy for being easy to cause depth data are relatively low.
As it can be seen that in existing 3D imaging technique, the depth that is obtained by the imaging scheme of RGB camera combination IR cameras
The data precision is relatively low.
Invention content
A kind of depth data acquisition methods of offer of the embodiment of the present invention and mobile terminal, to solve to pass through in the prior art
The relatively low problem of depth data accuracy that the imaging scheme of RGB camera combination IR cameras obtains.
In order to solve the above-mentioned technical problem, the invention is realized in this way：
In a first aspect, an embodiment of the present invention provides a kind of depth data acquisition methods, it is applied to mobile terminal, the side
Method includes：
The RGB data and the first IR cameras for obtaining the first RGB camera the image collected of the mobile terminal are adopted
The IR data of the image collected；
Using the RGB data and the IR data as the input data of target algorithm, pass through the target algorithm meter
Calculation obtains corresponding depth data；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training to multigroup training data
Data include the first RGB data, the first IR data and the first depth data under same scene.
Second aspect, an embodiment of the present invention provides a kind of mobile terminal, the mobile terminal includes：
Acquisition module, the RGB data for obtaining the first RGB camera the image collected of the mobile terminal and
The IR data of one IR camera the image collected；
Computing module, for using the RGB data and the IR data as the input data of target algorithm, passing through institute
It states target algorithm and corresponding depth data is calculated；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training to multigroup training data
Data include the first RGB data, the first IR data and the first depth data under same scene.
The third aspect, an embodiment of the present invention provides another mobile terminal, including processor, memory is stored in institute
The computer program that can be run on memory and on the processor is stated, when the computer program is executed by the processor
The step of realizing above-mentioned depth data acquisition methods.
Fourth aspect, an embodiment of the present invention provides a kind of computer readable storage medium, the computer-readable storage
Computer program is stored on medium, the computer program realizes above-mentioned depth data acquisition methods when being executed by processor
Step.
In this way, corresponding depth can be calculated according to RGB data, IR data and target algorithm in the embodiment of the present invention
Data, the target algorithm is to acquire training data in early period, between initial RGB data and IR data and depth data
Correspondence carries out the algorithm that deep learning obtains so that the depth data that the embodiment of the present invention is calculated is closer to reality
The depth data on border can effectively improve the depth data obtained by the imaging scheme of RGB camera combination IR cameras
Accuracy.In addition, the embodiment of the present invention need not use depth camera that can also obtain the higher depth data of accuracy, it can
Effectively save cost.
Description of the drawings
In order to illustrate the technical solution of the embodiments of the present invention more clearly, needed in being described below to the embodiment of the present invention
Attached drawing to be used is briefly described, it should be apparent that, drawings in the following description are only some embodiments of the invention,
For those of ordinary skill in the art, without having to pay creative labor, it can also obtain according to these attached drawings
Take other attached drawings.
Fig. 1 is a kind of flow chart of depth data acquisition methods provided in an embodiment of the present invention；
Fig. 2 is the flow chart of another depth data acquisition methods provided in an embodiment of the present invention；
Fig. 3 is one of the structure chart of mobile terminal provided in an embodiment of the present invention；
Fig. 4 is the two of the structure chart of mobile terminal provided in an embodiment of the present invention；
Fig. 5 is the hardware architecture diagram of mobile terminal provided in an embodiment of the present invention.
Specific implementation mode
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out clear, complete
Site preparation describes, it is clear that described embodiments are some of the embodiments of the present invention, instead of all the embodiments.Based on this hair
Embodiment in bright, those of ordinary skill in the art's acquired every other implementation without creative efforts
Example, shall fall within the protection scope of the present invention.
It is a kind of flow chart of depth data acquisition methods provided in an embodiment of the present invention, the depth referring to Fig. 1, Fig. 1
Data capture method is applied to mobile terminal, as shown in Figure 1, including the following steps：
Step 101, the RGB data and the first IR for obtaining described the first RGB camera the image collected of mobile terminal
The IR data of camera the image collected.
In the step, the method obtain the RGB data of the first RGB camera the image collected of the mobile terminal with
And the first IR camera the image collected IR data.The method can need the when of obtaining depth data to obtain the shifting
The RGB data of the first RGB camera the image collected of dynamic terminal and the IR data of the first IR camera the image collected.
It is understood that in inventing some embodiments, the method is obtaining the first RGB of the mobile terminal camera shootings
Before the RGB data of head the image collected and the IR data of the first IR camera the image collected, the shifting is also controlled
The first RGB cameras and the first IR camera synchronous acquisition images of dynamic terminal.It should be noted that the embodiment of the present invention
In, the first RGB cameras and the first IR cameras shooting scene it is consistent, i.e., the described first RGB cameras with
And the shooting angle and shooting direction of the first IR cameras are identical.
For example, it is assumed that user executes for triggering the intended application destination application on the mobile terminal
Program obtains the trigger action of depth data, such as opens the face identification functions etc. on the mobile terminal, and the method is rung
Trigger action described in Ying Yu controls the first RGB cameras and the first IR picture synchronization collection images of the mobile terminal.
Step 102, using the RGB data and the IR data as the input data of target algorithm, pass through the mesh
Corresponding depth data is calculated in mark algorithm, wherein the target algorithm is by carrying out engineering to multigroup training data
The algorithm that acquistion is arrived, every group of training data include the first RGB data, the first IR data and the first depth number under same scene
According to.
In the step, the method is led to using the RGB data and the IR data as the input data of target algorithm
It crosses the target algorithm and corresponding depth data is calculated.The target algorithm is by carrying out machine to multigroup training data
Learn obtained algorithm, every group of training data includes the first RGB data, the first IR data and the first depth under same scene
Data.The target algorithm can be that the mobile terminal first passes through the calculation obtained to the progress machine learning of multigroup training data in advance
Method, can also be by other-end (such as computer, laptop or other mobile terminals etc.) by multigroup trained number
It is obtained according to progress machine learning, and the algorithm sent to the mobile terminal, the embodiment of the present invention are not specifically limited this.
Specifically, the target algorithm can be by the first RGB data and first of every group of training data in multigroup training data
IR data carry out the algorithm that machine learning obtains as input data, the first depth data as output data.
Multigroup training data can be the 2nd RGB cameras and the first depth camera under multiple and different scenes
The data of the image of synchronous acquisition, accordingly, first RGB data are the number of the image of the 2nd RGB cameras acquisition
According to the first IR data are the data for the image that the IR cameras of first depth camera acquire, first depth
Data are the data of the image of first depth camera acquisition.The 2nd RGB cameras and the first RGB cameras
Configuration and parameter all same, the configuration of the IR cameras of first depth camera and the first IR cameras and parameter
All same.It is understood that the 2nd RGB cameras can be that the same RGB is imaged with the first RGB cameras
Head, or two RGB cameras of configuration and parameter all same, that is to say, that the method can directly use described
First RGB cameras acquire training data, can also use and its of the configuration of the first RGB cameras and parameter all same
He acquires training data by RGB cameras.First depth camera is the depth camera for including IR cameras, such as structure
Light (Structured Light) camera or TOF (Time of Flight, flight time telemetry) camera, described
The configuration and parameter all same of the configuration of the IR cameras of one depth camera and parameter with the first IR cameras.
Multigroup training data can be that the 2nd RGB cameras, the 2nd IR cameras and the second depth camera exist
The data of the image of synchronous acquisition under multiple and different scenes, the configuration of the 2nd RGB cameras and the first RGB cameras
And parameter all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.Similarly, described
2nd IR cameras and the first IR cameras can be the same IR cameras, or configure and parameter all same
Two IR cameras, i.e. the method can directly use the first IR cameras acquire training data, can also use with
The configuration of the first IR data and other IR cameras of parameter all same acquire training data.It should be noted that acquisition
When training data, the 2nd RGB cameras, the 2nd IR cameras and the second depth camera are in the shooting field of synchronization
Scape is consistent, i.e., the described 2nd RGB cameras, the 2nd IR cameras and the second depth camera shooting angle and shooting side
To consistent.Second depth camera can be any depth camera that can calculate depth data, such as can be knot
Structure light video camera head can also be TOF cameras, can also be binocular camera.
In the embodiment of the present invention, above-mentioned mobile terminal can be any mobile terminal, such as：Mobile phone, tablet computer
(Tablet Personal Computer), laptop computer (Laptop Computer), personal digital assistant (personal
Digital assistant, abbreviation PDA), mobile Internet access device (Mobile Internet Device, MID) or wearable
Equipment (Wearable Device) etc..
In the present embodiment, the depth data acquisition methods can be calculated according to RGB data, IR data and target algorithm
Obtain corresponding depth data, the target algorithm is to acquire training data in early period, to initial RGB data and IR data with
Correspondence between depth data carries out the algorithm that deep learning obtains so that the depth number that the embodiment of the present invention is calculated
According to actual depth data is closer to, it can effectively improve and be obtained by the imaging scheme of RGB camera combination IR cameras
The accuracy of the depth data arrived.In addition, the embodiment of the present invention need not use depth camera, also to obtain accuracy higher
Depth data, can effectively save cost.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is the 2nd RGB cameras and the first depth camera in multiple and different scenes
The data of the image of lower synchronous acquisition, the first IR data in every group of training data are that the IR of first depth camera is imaged
The data of the image of head acquisition, configuration and parameter all same of the 2nd RGB cameras with the first RGB cameras, institute
State the IR cameras of the first depth camera and the configuration of the first IR cameras and parameter all same.
In the embodiment, described by carrying out machine learning to multigroup training data, including will be in multigroup training data
The first RGB data and the first IR data of every group of training data as input data, the first depth data as output data,
Carry out machine learning.That is, the target algorithm is by the first RGB numbers of every group of training data in multigroup training data
According to this and the first IR data are as input data, and the first depth data carries out the algorithm that machine learning obtains as output data.
Specifically, the mode for obtaining multigroup training data can be the 2nd RGB cameras of control and the first depth camera
Head synchronous acquisition image under multiple and different scenes, then obtains the 2nd RGB cameras, first depth camera
The data of IR cameras and the first depth camera image of synchronous acquisition under multiple and different scenes.Correspondingly, institute
The data for the image that the first RGB data is the 2nd RGB cameras acquisition are stated, the first IR data are first depth
The data of the image of the IR cameras acquisition of camera, first depth data are the figure of first depth camera acquisition
The data of picture.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is that the 2nd RGB cameras, the 2nd IR cameras and the second depth camera exist
The data of the image of synchronous acquisition under multiple and different scenes, the configuration of the 2nd RGB cameras and the first RGB cameras
And parameter all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.
In the embodiment, described by carrying out machine learning to multigroup training data, including will be in multigroup training data
The first RGB data and the first IR data of every group of training data as input data, the first depth data as output data,
Carry out machine learning.That is, the target algorithm is by the first RGB numbers of every group of training data in multigroup training data
According to this and the first IR data are as input data, and the first depth data carries out the algorithm that machine learning obtains as output data.
Specifically, the mode for obtaining multigroup training data can be control the 2nd RGB cameras, the 2nd IR cameras and
Then second depth camera synchronous acquisition image under multiple and different scenes obtains the 2nd RGB cameras, the 2nd IR takes the photograph
The data of the image of synchronous acquisition under multiple and different scenes as head and the second depth camera.Correspondingly, the first RGB
Data are the data of the image of the 2nd RGB cameras acquisition, and the first IR data acquire for the 2nd IR cameras
Image data, first depth data be second depth camera acquisition image data.
In the embodiment of the present invention, the different scene may include different objects under same background, different background
Under same object, the different objects under different background, with same background same object separated by different distances in one kind or
Person is a variety of, and the embodiment of the present invention is not specifically limited this.It is understood that the scene of multigroup training data covering is got over
More, the accuracy for carrying out the algorithm that machine learning obtains is higher.
Optionally, the target algorithm obtains target depth data to calculate the second RGB data and the 2nd IR data, and
Error between the target depth data and the second depth data falls into the algorithm of default error range；
Wherein, second RGB data, the 2nd IR data and the second depth data are the 2nd RGB cameras, the 2nd IR
The data of the image of camera and depth camera synchronous acquisition.
In the embodiment, the target algorithm is that the second RGB data of calculating and the 2nd IR data obtain target depth number
According to, and the error between the target depth data and the second depth data falls into the algorithm of default error range.Target algorithm
Generating process in, can constantly test to the algorithm of generation, by calculate error fall into default error range algorithm make
For target algorithm.
Specifically, can using in every group of training data the first RGB data and the first IR data as input data,
First depth data carries out machine learning, obtains the first algorithm as output data；Then the 2nd RGB cameras, first are obtained
The data of the image of the IR cameras of depth camera and the first depth camera synchronous acquisition, or obtain the 2nd RGB and take the photograph
As the data of the image of head, the 2nd IR cameras and the second depth camera synchronous acquisition, at least one set of inspection data is obtained,
Every group of inspection data includes the second RGB data, the 2nd IR data and the second depth data.By second in every group of inspection data
The corresponding mesh of this group of inspection data is calculated in the input data of RGB data and the 2nd IR data as first algorithm
Depth data is marked, and in the case that the error between target depth data and the second depth data falls into default error range,
Using first algorithm as target algorithm；Otherwise, i.e., between the target depth data and second depth data
When error does not fall within default error range, continue to obtain training data progress machine learning.
Calculate the error between the target depth data and the second depth data of this group of inspection data of one group of inspection data
Mode, can calculate difference between the target depth value of pixel and the second depth value of the pixel on image, so
Afterwards according to the error of this group of inspection data of mathematic interpolation of multiple pixels on image.Such as calculate the difference of multiple pixels
Error of the average value as this group of inspection data, either the variance of the difference of the multiple pixels of calculating or the difference of two squares are as the group
The error of inspection data.Multiple pixels may include all pixels point on image in described image, can also be only to include
Multiple pixels of the main part of image.
It is understood that when inspection data is multigroup, error that can be based on every group of inspection data is preset with described
Error range is compared respectively, if the error of at least one set of inspection data in multigroup inspection data does not fall within described preset
Error range, it is determined that the error of first algorithm does not fall within default error range.Multigroup inspection data can also be made
For one it is whole be compared with the default error range, such as calculate the average value of the error of multigroup inspection data, then
The average value of the error of multigroup inspection data is compared with the default error range, if the error of multigroup inspection data
Average value does not fall within the default error range, it is determined that the error of first algorithm does not fall within default error range.
In this way, being tested by the algorithm obtained to study so that the error of obtained target algorithm falls into default mistake
Poor range, so as to strictly control the image being calculated by target algorithm depth data error, improve depth number
According to accuracy.
Optionally, the data of the image of the synchronous acquisition include：
It is identical in acquisition frame rate, the data of the image of synchronous acquisition.
In the embodiment, the data of the image of the synchronous acquisition be specifically included in acquisition frame rate it is identical in the case of, together
Walk the data of the image of acquisition.In such manner, it is possible to ensure the 2nd RGB cameras, the 2nd IR cameras and depth camera three
Synchronous acquisition image effectively avoids the Algorithm Error for causing study to obtain because three is asynchronous from increasing, so as to be effectively ensured
The precision for the target algorithm that machine learning obtains.
It is the flow chart of another depth data acquisition methods provided in an embodiment of the present invention, the side referring to Fig. 2, Fig. 2
Method is applied to mobile terminal, as shown in Fig. 2, including the following steps：
Step 201 receives trigger action of the user to destination application.
In the step, the method receives trigger action of the user to destination application.The destination application can
Think the application program for needing to obtain depth data, such as realizing the application program of recognition of face, the trigger action can
Think the trigger action for obtaining depth data for triggering the destination application, such as opens the face of destination application
Identification function.
Step 202, in response to the trigger action, control the first RGB cameras and the first IR of the mobile terminal
Camera synchronous acquisition image.
In the step, the method in response to the trigger action, control the first RGB cameras of the mobile terminal with
And the first IR camera synchronous acquisition images.The synchronous acquisition image is identical in acquisition frame rate, in same a period of time
Carve acquisition image.
Step 203, the RGB data and the first IR for obtaining described the first RGB camera the image collected of mobile terminal
The IR data of camera the image collected.
Step 204, using the RGB data and the IR data as the input data of target algorithm, pass through the mesh
Corresponding depth data is calculated in mark algorithm, wherein the target algorithm is by carrying out engineering to multigroup training data
The algorithm that acquistion is arrived, every group of training data include the first RGB data, the first IR data and the first depth number under same scene
According to.
The step 203 and step 204 and the step 101 and step 102 phase in present invention embodiment shown in FIG. 1
Together, details are not described herein again.
Step 205, to the destination application send described in the depth data that is calculated.
In the step, the method by the target algorithm after being calculated corresponding depth data, to the mesh
The depth data being calculated described in mark application program transmission, in this way, the destination application can be according to the depth number
Factually existing corresponding function.
In the present embodiment, when the depth data acquisition methods destination application needs to obtain depth data, according to RGB
Corresponding depth data is calculated in data, IR data and target algorithm, and the target algorithm is to acquire training number in early period
According to the correspondence between initial RGB data and IR data and depth data carries out the algorithm that deep learning obtains so that
The depth data that the embodiment of the present invention is calculated is closer to actual depth data, can effectively improve and be taken the photograph by RGB
As the accuracy for the depth data that the imaging scheme of head combination IR cameras obtains.In addition, the embodiment of the present invention need not use
Depth camera can also obtain the higher depth data of accuracy, can effectively save cost.
It is one of the structure chart of mobile terminal provided in an embodiment of the present invention referring to Fig. 3, Fig. 3, as shown in figure 3, mobile whole
End 300 includes：
Acquisition module 301, for obtain the RGB data of the first RGB camera the image collected of the mobile terminal with
And the first IR camera the image collected IR data；
Computing module 302, for using the RGB data and the IR data as the input data of target algorithm, leading to
It crosses the target algorithm and corresponding depth data is calculated；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training to multigroup training data
Data include the first RGB data, the first IR data and the first depth data under same scene.
Optionally, be referring to Fig. 4, Fig. 4 mobile terminal provided in an embodiment of the present invention structure chart two, as shown in figure 4,
The mobile terminal 300 further includes：
Receiving module 303, for receiving trigger action of the user to destination application；
Control module 304, in response to the trigger action, control the first RGB cameras of the mobile terminal with
And the first IR camera synchronous acquisition images；
Sending module 305, for the depth data that is calculated described in being sent to the destination application.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is the 2nd RGB cameras and the first depth camera in multiple and different scenes
The data of the image of lower synchronous acquisition, the first IR data in every group of training data are that the IR of first depth camera is imaged
The data of the image of head acquisition, configuration and parameter all same of the 2nd RGB cameras with the first RGB cameras, institute
State the IR cameras of the first depth camera and the configuration of the first IR cameras and parameter all same.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is that the 2nd RGB cameras, the 2nd IR cameras and the second depth camera exist
The data of the image of synchronous acquisition under multiple and different scenes, the configuration of the 2nd RGB cameras and the first RGB cameras
And parameter all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.
Optionally, the data of the image of the synchronous acquisition be included in acquisition frame rate it is identical in the case of, synchronous acquisition
The data of image.
Correspondence can be calculated according to RGB data, IR data and target algorithm in mobile terminal provided in this embodiment
Depth data, the target algorithm be early period acquire training data, to initial RGB data and IR data and depth data
Between correspondence carry out the obtained algorithm of deep learning so that the depth data that the embodiment of the present invention is calculated more adjunction
It is bordering on actual depth data, the depth obtained by the imaging scheme of RGB camera combination IR cameras can be effectively improved
The accuracy of data.In addition, the embodiment of the present invention need not use depth camera that can also obtain the higher depth number of accuracy
According to can effectively save cost.
The hardware architecture diagram of Fig. 5 a kind of mobile terminals that embodiment provides to realize the present invention, as shown in figure 5, should
Mobile terminal 500 includes but not limited to：Radio frequency unit 501, network module 502, audio output unit 503, input unit 504,
Sensor 505, display unit 506, user input unit 507, interface unit 508, memory 509, processor 510, Yi Ji electricity
The components such as source 511.It will be understood by those skilled in the art that mobile terminal structure shown in Fig. 5 is not constituted to mobile terminal
Restriction, mobile terminal may include either combining certain components or different components than illustrating more or fewer components
Arrangement.In embodiments of the present invention, mobile terminal include but not limited to mobile phone, tablet computer, laptop, palm PC,
Car-mounted terminal, wearable device and pedometer etc..
Wherein, processor 510 are used for：
The RGB data and the first IR cameras for obtaining the first RGB camera the image collected of the mobile terminal are adopted
The IR data of the image collected；
Using the RGB data and the IR data as the input data of target algorithm, pass through the target algorithm meter
Calculation obtains corresponding depth data；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training to multigroup training data
Data include the first RGB data, the first IR data and the first depth data under same scene.
Optionally, the processor 510, which executes, obtains described the first RGB camera the image collected of mobile terminal
Before the IR data of RGB data and the first IR camera the image collected, following steps are can also be achieved：
Receive trigger action of the user to destination application；
In response to the trigger action, the first RGB cameras and the first IR cameras for controlling the mobile terminal are same
Step acquisition image；
It is described corresponding depth data is calculated by the target algorithm after, the method further includes：
To the depth data being calculated described in destination application transmission.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is the 2nd RGB cameras and the first depth camera in multiple and different scenes
The data of the image of lower synchronous acquisition, the first IR data in every group of training data are that the IR of first depth camera is imaged
The data of the image of head acquisition, configuration and parameter all same of the 2nd RGB cameras with the first RGB cameras, institute
State the IR cameras of the first depth camera and the configuration of the first IR cameras and parameter all same.
Optionally, it is described by multigroup training data carry out machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input number
According to the first depth data carries out machine learning as output data；
Wherein, multigroup training data is that the 2nd RGB cameras, the 2nd IR cameras and the second depth camera exist
The data of the image of synchronous acquisition under multiple and different scenes, the configuration of the 2nd RGB cameras and the first RGB cameras
And parameter all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.
Optionally, the data of the image of the synchronous acquisition include：
It is identical in acquisition frame rate, the data of the image of synchronous acquisition.
In the embodiment of the present invention, correspondence can be calculated according to RGB data, IR data and target algorithm in mobile terminal
Depth data, the target algorithm be early period acquire training data, to initial RGB data and IR data and depth data
Between correspondence carry out the obtained algorithm of deep learning so that the depth data that the embodiment of the present invention is calculated more adjunction
It is bordering on actual depth data, the depth obtained by the imaging scheme of RGB camera combination IR cameras can be effectively improved
The accuracy of data.In addition, the embodiment of the present invention need not use depth camera that can also obtain the higher depth number of accuracy
According to can effectively save cost.
It should be understood that the embodiment of the present invention in, radio frequency unit 501 can be used for receiving and sending messages or communication process in, signal
Send and receive, specifically, by from base station downlink data receive after, to processor 510 handle；In addition, by uplink
Data are sent to base station.In general, radio frequency unit 501 includes but not limited to antenna, at least one amplifier, transceiver, coupling
Device, low-noise amplifier, duplexer etc..In addition, radio frequency unit 501 can also by radio communication system and network and other set
Standby communication.
Mobile terminal has provided wireless broadband internet to the user by network module 502 and has accessed, and such as user is helped to receive
Send e-mails, browse webpage and access streaming video etc..
It is that audio output unit 503 can receive radio frequency unit 501 or network module 502 or in memory 509
The audio data of storage is converted into audio signal and exports to be sound.Moreover, audio output unit 503 can also be provided and be moved
The relevant audio output of specific function that dynamic terminal 500 executes is (for example, call signal receives sound, message sink sound etc.
Deng).Audio output unit 503 includes loud speaker, buzzer and receiver etc..
Input unit 504 is for receiving audio or video signal.Input unit 504 may include graphics processor
(Graphics Processing Unit, GPU) 5041 and microphone 5042, graphics processor 5041 is in video acquisition mode
Or the image data of the static images or video obtained by image capture apparatus (such as camera) in image capture mode carries out
Reason.Treated, and picture frame may be displayed on display unit 506.Through graphics processor 5041, treated that picture frame can be deposited
Storage is sent in memory 509 (or other storage mediums) or via radio frequency unit 501 or network module 502.Mike
Wind 5042 can receive sound, and can be audio data by such acoustic processing.Treated audio data can be
The format output of mobile communication base station can be sent to via radio frequency unit 501 by being converted in the case of telephone calling model.
Mobile terminal 500 further includes at least one sensor 505, such as optical sensor, motion sensor and other biographies
Sensor.Specifically, optical sensor includes ambient light sensor and proximity sensor, wherein ambient light sensor can be according to environment
The light and shade of light adjusts the brightness of display panel 5061, and proximity sensor can close when mobile terminal 500 is moved in one's ear
Display panel 5061 and/or backlight.As a kind of motion sensor, accelerometer sensor can detect in all directions (general
For three axis) size of acceleration, size and the direction of gravity are can detect that when static, can be used to identify mobile terminal posture (ratio
Such as horizontal/vertical screen switching, dependent game, magnetometer pose calibrating), Vibration identification correlation function (such as pedometer, tap)；It passes
Sensor 505 can also include fingerprint sensor, pressure sensor, iris sensor, molecule sensor, gyroscope, barometer, wet
Meter, thermometer, infrared sensor etc. are spent, details are not described herein.
Display unit 506 is for showing information input by user or being supplied to the information of user.Display unit 506 can wrap
Display panel 5061 is included, liquid crystal display (Liquid Crystal Display, LCD), Organic Light Emitting Diode may be used
Forms such as (Organic Light-Emitting Diode, OLED) configure display panel 5061.
User input unit 507 can be used for receiving the number or character information of input, and generate the use with mobile terminal
Family is arranged and the related key signals input of function control.Specifically, user input unit 507 include touch panel 5071 and
Other input equipments 5072.Touch panel 5071, also referred to as touch screen collect user on it or neighbouring touch operation
(for example user uses any suitable objects or attachment such as finger, stylus on touch panel 5071 or in touch panel 5071
Neighbouring operation).Touch panel 5071 may include both touch detecting apparatus and touch controller.Wherein, touch detection
Device detects the touch orientation of user, and detects the signal that touch operation is brought, and transmits a signal to touch controller；Touch control
Device processed receives touch information from touch detecting apparatus, and is converted into contact coordinate, then gives processor 510, receiving area
It manages the order that device 510 is sent and is executed.Furthermore, it is possible to more using resistance-type, condenser type, infrared ray and surface acoustic wave etc.
Type realizes touch panel 5071.In addition to touch panel 5071, user input unit 507 can also include other input equipments
5072.Specifically, other input equipments 5072 can include but is not limited to physical keyboard, function key (such as volume control button,
Switch key etc.), trace ball, mouse, operating lever, details are not described herein.
Further, touch panel 5071 can be covered on display panel 5061, when touch panel 5071 is detected at it
On or near touch operation after, send processor 510 to determine the type of touch event, be followed by subsequent processing device 510 according to touch
The type for touching event provides corresponding visual output on display panel 5061.Although in Figure 5, touch panel 5071 and display
Panel 5061 is to realize the function that outputs and inputs of mobile terminal as two independent components, but in some embodiments
In, can be integrated by touch panel 5071 and display panel 5061 and realize the function that outputs and inputs of mobile terminal, it is specific this
Place does not limit.
Interface unit 508 is the interface that external device (ED) is connect with mobile terminal 500.For example, external device (ED) may include having
Line or wireless head-band earphone port, external power supply (or battery charger) port, wired or wireless data port, storage card end
Mouth, port, the port audio input/output (I/O), video i/o port, earphone end for connecting the device with identification module
Mouthful etc..Interface unit 508 can be used for receiving the input (for example, data information, electric power etc.) from external device (ED) and
By one or more elements that the input received is transferred in mobile terminal 500 or can be used in 500 He of mobile terminal
Transmission data between external device (ED).
Memory 509 can be used for storing software program and various data.Memory 509 can include mainly storing program area
And storage data field, wherein storing program area can storage program area, application program (such as the sound needed at least one function
Sound playing function, image player function etc.) etc.；Storage data field can store according to mobile phone use created data (such as
Audio data, phone directory etc.) etc..In addition, memory 509 may include high-speed random access memory, can also include non-easy
The property lost memory, a for example, at least disk memory, flush memory device or other volatile solid-state parts.
Processor 510 is the control centre of mobile terminal, utilizes each of various interfaces and the entire mobile terminal of connection
A part by running or execute the software program and/or module that are stored in memory 509, and calls and is stored in storage
Data in device 509 execute the various functions and processing data of mobile terminal, to carry out integral monitoring to mobile terminal.Place
Reason device 510 may include one or more processing units；Preferably, processor 510 can integrate application processor and modulatedemodulate is mediated
Manage device, wherein the main processing operation system of application processor, user interface and application program etc., modem processor is main
Processing wireless communication.It is understood that above-mentioned modem processor can not also be integrated into processor 510.
Mobile terminal 500 can also include the power supply 511 (such as battery) powered to all parts, it is preferred that power supply 511
Can be logically contiguous by power-supply management system and processor 510, to realize management charging by power-supply management system, put
The functions such as electricity and power managed.
In addition, mobile terminal 500 includes some unshowned function modules, details are not described herein.
Preferably, the embodiment of the present invention also provides a kind of mobile terminal, including processor 510, and memory 509 is stored in
On memory 509 and the computer program that can be run on the processor 510, the computer program are executed by processor 510
Each process of the above-mentioned depth data acquisition methods embodiments of Shi Shixian, and identical technique effect can be reached, to avoid repeating,
Which is not described herein again.
The embodiment of the present invention also provides a kind of computer readable storage medium, and meter is stored on computer readable storage medium
Calculation machine program, the computer program realize each process of above-mentioned depth data acquisition methods embodiment when being executed by processor,
And identical technique effect can be reached, to avoid repeating, which is not described herein again.Wherein, the computer readable storage medium,
Such as read-only memory (Read-Only Memory, abbreviation ROM), random access memory (Random Access Memory, letter
Claim RAM), magnetic disc or CD etc..
It should be noted that herein, the terms "include", "comprise" or its any other variant are intended to non-row
His property includes, so that process, method, article or device including a series of elements include not only those elements, and
And further include other elements that are not explicitly listed, or further include for this process, method, article or device institute it is intrinsic
Element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that including this
There is also other identical elements in the process of element, method, article or device.
Through the above description of the embodiments, those skilled in the art can be understood that above-described embodiment side
Method can add the mode of required general hardware platform to realize by software, naturally it is also possible to by hardware, but in many cases
The former is more preferably embodiment.Based on this understanding, technical scheme of the present invention substantially in other words does the prior art
Going out the part of contribution can be expressed in the form of software products, which is stored in a storage medium
In (such as ROM/RAM, magnetic disc, CD), including some instructions are used so that a station terminal (can be mobile phone, computer, service
Device, air conditioner or network equipment etc.) execute method described in each embodiment of the present invention.
The above description is merely a specific embodiment, but scope of protection of the present invention is not limited thereto, any
Those familiar with the art in the technical scope disclosed by the present invention, can easily think of the change or the replacement, and should all contain
Lid is within protection scope of the present invention.Therefore, protection scope of the present invention should be subject to the protection scope in claims.
Claims (11)
1. a kind of depth data acquisition methods are applied to mobile terminal, which is characterized in that the method includes：
The RGB data and the first infrared IR cameras for obtaining the first RGB camera the image collected of the mobile terminal are adopted
The IR data of the image collected；
Using the RGB data and the IR data as the input data of target algorithm, calculated by the target algorithm
To corresponding depth data；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training data to multigroup training data
Including the first RGB data, the first IR data and the first depth data under same scene.
2. depth data acquisition methods as described in claim 1, which is characterized in that described to obtain the mobile terminal first
Before the IR data of the RGB data of RGB camera the image collected and the first IR camera the image collected, the side
Method further includes：
Receive trigger action of the user to destination application；
In response to the trigger action, the first RGB cameras and the first IR cameras that control the mobile terminal are synchronized and are adopted
Collect image；
It is described corresponding depth data is calculated by the target algorithm after, the method further includes：
To the depth data being calculated described in destination application transmission.
3. depth data acquisition methods as described in claim 1, which is characterized in that described by being carried out to multigroup training data
Machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input data,
One depth data carries out machine learning as output data；
Wherein, multigroup training data be the 2nd RGB cameras and the first depth camera multiple and different scenes similarly hereinafter
The data of the image of acquisition are walked, the first IR data in every group of training data are that the IR cameras of first depth camera are adopted
The data of the image of collection, the configuration of the 2nd RGB cameras and the first RGB cameras and parameter all same, described the
Configuration and parameter all same of the IR cameras of one depth camera with the first IR cameras.
4. depth data acquisition methods as described in claim 1, which is characterized in that described by being carried out to multigroup training data
Machine learning, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input data,
One depth data carries out machine learning as output data；
Wherein, multigroup training data is the 2nd RGB cameras, the 2nd IR cameras and the second depth camera multiple
The data of the image of synchronous acquisition under different scenes, configuration and ginseng of the 2nd RGB cameras with the first RGB cameras
Number all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.
5. depth data acquisition methods as described in claim 3 or 4, which is characterized in that the number of the image of the synchronous acquisition
According to including：
It is identical in acquisition frame rate, the data of the image of synchronous acquisition.
6. a kind of mobile terminal, which is characterized in that the mobile terminal includes：
Acquisition module, the RGB data and first for obtaining the first RGB camera the image collected of the mobile terminal are red
The IR data of outer IR cameras the image collected；
Computing module, for using the RGB data and the IR data as the input data of target algorithm, passing through the mesh
Corresponding depth data is calculated in mark algorithm；
Wherein, the target algorithm is by carrying out the algorithm that machine learning obtains, every group of training data to multigroup training data
Including the first RGB data, the first IR data and the first depth data under same scene.
7. mobile terminal as claimed in claim 6, which is characterized in that the mobile terminal further includes：
Receiving module, for receiving trigger action of the user to destination application；
Control module, in response to the trigger action, controlling the first RGB cameras and the first IR of the mobile terminal
Camera synchronous acquisition image；
Sending module, for the depth data that is calculated described in being sent to the destination application.
8. mobile terminal as claimed in claim 6, which is characterized in that described by carrying out engineering to multigroup training data
It practises, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input data,
One depth data carries out machine learning as output data；
Wherein, multigroup training data be the 2nd RGB cameras and the first depth camera multiple and different scenes similarly hereinafter
The data of the image of acquisition are walked, the first IR data in every group of training data are that the IR cameras of first depth camera are adopted
The data of the image of collection, the configuration of the 2nd RGB cameras and the first RGB cameras and parameter all same, described the
Configuration and parameter all same of the IR cameras of one depth camera with the first IR cameras.
9. mobile terminal as claimed in claim 6, which is characterized in that described by carrying out engineering to multigroup training data
It practises, including：
Using the first RGB data of every group of training data in multigroup training data and the first IR data as input data,
One depth data carries out machine learning as output data；
Wherein, multigroup training data is the 2nd RGB cameras, the 2nd IR cameras and the second depth camera multiple
The data of the image of synchronous acquisition under different scenes, configuration and ginseng of the 2nd RGB cameras with the first RGB cameras
Number all same, configuration and parameter all same of the 2nd IR cameras with the first IR cameras.
10. mobile terminal as claimed in claim 8 or 9, which is characterized in that the data of the image of the synchronous acquisition are included in
In the case of acquisition frame rate is identical, the data of the image of synchronous acquisition.
11. a kind of mobile terminal, which is characterized in that including processor, memory is stored on the memory and can be described
The computer program run on processor is realized when the computer program is executed by the processor as in claim 1 to 5
The step of any one of them depth data acquisition methods.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810325685.5A CN108564613A (en) | 2018-04-12 | 2018-04-12 | A kind of depth data acquisition methods and mobile terminal |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810325685.5A CN108564613A (en) | 2018-04-12 | 2018-04-12 | A kind of depth data acquisition methods and mobile terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108564613A true CN108564613A (en) | 2018-09-21 |
Family
ID=63534796
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810325685.5A Pending CN108564613A (en) | 2018-04-12 | 2018-04-12 | A kind of depth data acquisition methods and mobile terminal |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108564613A (en) |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109544616A (en) * | 2018-12-11 | 2019-03-29 | 维沃移动通信有限公司 | A kind of depth information determines method and terminal |
| CN109615648A (en) * | 2018-12-07 | 2019-04-12 | 深圳前海微众银行股份有限公司 | Depth of field data conversion method, device, equipment and computer readable storage medium |
| CN112912896A (en) * | 2018-12-14 | 2021-06-04 | 苹果公司 | Machine learning assisted image prediction |
| CN112967328A (en) * | 2021-03-20 | 2021-06-15 | 杭州知存智能科技有限公司 | Image depth information local dynamic generation method and device |
| WO2023103559A1 (en) * | 2021-12-07 | 2023-06-15 | 中兴通讯股份有限公司 | Three-dimensional recognition device, terminal, calibration method, and storage medium |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467661A (en) * | 2010-11-11 | 2012-05-23 | Lg电子株式会社 | Multimedia device and method for controlling the same |
| WO2014172226A1 (en) * | 2013-04-15 | 2014-10-23 | Microsoft Corporation | Multimodal foreground background segmentation |
| WO2014172228A1 (en) * | 2013-04-15 | 2014-10-23 | Microsoft Corporation | Robust stereo depth system |
| WO2016018393A1 (en) * | 2014-07-31 | 2016-02-04 | Hewlett-Packard Development Company, Lp | Touch region projection onto a touch-sensitive surface |
| EP3101893A1 (en) * | 2014-01-29 | 2016-12-07 | LG Innotek Co., LTD. | Depth information extracting device and method |
| CN206470834U (en) * | 2016-12-09 | 2017-09-05 | 深圳奥比中光科技有限公司 | A kind of device for obtaining target depth image |
| CN107407959A (en) * | 2014-09-30 | 2017-11-28 | 惠普发展公司，有限责任合伙企业 | The manipulation of 3-D view based on posture |
| CN107466411A (en) * | 2015-04-14 | 2017-12-12 | 微软技术许可有限责任公司 | Two-dimensional infrared depth sense |
| CN107862259A (en) * | 2017-10-24 | 2018-03-30 | 重庆虚拟实境科技有限公司 | Human image collecting method and device, terminal installation and computer-readable recording medium |
-
2018
- 2018-04-12 CN CN201810325685.5A patent/CN108564613A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102467661A (en) * | 2010-11-11 | 2012-05-23 | Lg电子株式会社 | Multimedia device and method for controlling the same |
| WO2014172226A1 (en) * | 2013-04-15 | 2014-10-23 | Microsoft Corporation | Multimodal foreground background segmentation |
| WO2014172228A1 (en) * | 2013-04-15 | 2014-10-23 | Microsoft Corporation | Robust stereo depth system |
| EP3101893A1 (en) * | 2014-01-29 | 2016-12-07 | LG Innotek Co., LTD. | Depth information extracting device and method |
| WO2016018393A1 (en) * | 2014-07-31 | 2016-02-04 | Hewlett-Packard Development Company, Lp | Touch region projection onto a touch-sensitive surface |
| CN107407959A (en) * | 2014-09-30 | 2017-11-28 | 惠普发展公司，有限责任合伙企业 | The manipulation of 3-D view based on posture |
| CN107466411A (en) * | 2015-04-14 | 2017-12-12 | 微软技术许可有限责任公司 | Two-dimensional infrared depth sense |
| CN206470834U (en) * | 2016-12-09 | 2017-09-05 | 深圳奥比中光科技有限公司 | A kind of device for obtaining target depth image |
| CN107862259A (en) * | 2017-10-24 | 2018-03-30 | 重庆虚拟实境科技有限公司 | Human image collecting method and device, terminal installation and computer-readable recording medium |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109615648A (en) * | 2018-12-07 | 2019-04-12 | 深圳前海微众银行股份有限公司 | Depth of field data conversion method, device, equipment and computer readable storage medium |
| CN109544616A (en) * | 2018-12-11 | 2019-03-29 | 维沃移动通信有限公司 | A kind of depth information determines method and terminal |
| CN112912896A (en) * | 2018-12-14 | 2021-06-04 | 苹果公司 | Machine learning assisted image prediction |
| US11915460B2 (en) | 2018-12-14 | 2024-02-27 | Apple Inc. | Machine learning assisted image prediction |
| CN112967328A (en) * | 2021-03-20 | 2021-06-15 | 杭州知存智能科技有限公司 | Image depth information local dynamic generation method and device |
| WO2023103559A1 (en) * | 2021-12-07 | 2023-06-15 | 中兴通讯股份有限公司 | Three-dimensional recognition device, terminal, calibration method, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108564613A (en) | A kind of depth data acquisition methods and mobile terminal | |
| CN108540724A (en) | A kind of image pickup method and mobile terminal | |
| CN107483836B (en) | A kind of image pickup method and mobile terminal | |
| CN108495029A (en) | A kind of photographic method and mobile terminal | |
| CN107682639B (en) | A kind of image processing method, device and mobile terminal | |
| CN108307109A (en) | A kind of high dynamic range images method for previewing and terminal device | |
| CN109688322A (en) | A kind of method, device and mobile terminal generating high dynamic range images | |
| CN109348020A (en) | A kind of photographic method and mobile terminal | |
| CN107566749A (en) | Image pickup method and mobile terminal | |
| CN108320263A (en) | A kind of method, device and mobile terminal of image procossing | |
| CN108347558A (en) | A kind of method, apparatus and mobile terminal of image optimization | |
| CN110213485A (en) | A kind of image processing method and terminal | |
| CN113365085B (en) | Live video generation method and device | |
| CN108881544A (en) | A kind of method taken pictures and mobile terminal | |
| CN108683850A (en) | A kind of shooting reminding method and mobile terminal | |
| CN108462826A (en) | A kind of method and mobile terminal of auxiliary photo-taking | |
| CN109544486A (en) | A kind of image processing method and terminal device | |
| CN108881721A (en) | A kind of display methods and terminal | |
| CN108718389A (en) | A kind of screening-mode selection method and mobile terminal | |
| CN107959755A (en) | A kind of photographic method and mobile terminal | |
| CN110086998A (en) | A kind of image pickup method and terminal | |
| CN109358913A (en) | A kind of the starting method and terminal device of application program | |
| CN108833791A (en) | A kind of image pickup method and device | |
| CN108551562A (en) | A kind of method and mobile terminal of video communication | |
| CN108391011A (en) | A kind of face identification method and mobile terminal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20180921 |
|
| RJ01 | Rejection of invention patent application after publication |