WO2019218878A1 - 拍照修复方法以及装置、存储介质及终端设备 - Google Patents

拍照修复方法以及装置、存储介质及终端设备 Download PDF

Info

Publication number
WO2019218878A1
WO2019218878A1 PCT/CN2019/085452 CN2019085452W WO2019218878A1 WO 2019218878 A1 WO2019218878 A1 WO 2019218878A1 CN 2019085452 W CN2019085452 W CN 2019085452W WO 2019218878 A1 WO2019218878 A1 WO 2019218878A1
Authority
WO
WIPO (PCT)
Prior art keywords
dimensional
missing
torn
data
defective
Prior art date
Application number
PCT/CN2019/085452
Other languages
English (en)
French (fr)
Inventor
刘耀勇
陈岩
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2019218878A1 publication Critical patent/WO2019218878A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/77—Retouching; Inpainting; Scratch removal

Definitions

  • the embodiment of the present invention relates to the technical field of a terminal device, for example, to a camera repairing method and device, a storage medium, and a terminal device.
  • a photographing method, device, storage medium and terminal device provided by the embodiments of the present application can optimize related photographing technologies.
  • the embodiment of the present application provides a photo repairing method, including:
  • Feature repair is performed on the incomplete position on the missing object.
  • the embodiment of the present application provides a photo repairing apparatus, including:
  • a three-dimensional data acquisition module configured to acquire, by the recognition camera, three-dimensional data of the object of the torn object when the recognition camera captures a defective object
  • a position determining module configured to identify the three-dimensional data of the object by a preset defect recognition model to determine a missing position on the defective object
  • a repair processing module configured to perform feature repair on the defective position on the torn object.
  • the embodiment of the present application provides a computer readable storage medium, where a computer program is stored, and when the program is executed by the processor, the photo repairing method as described in the embodiment of the present application is implemented.
  • an embodiment of the present application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and operable on the processor, where the processor executes the computer program to implement the implementation of the present application.
  • a terminal device including a memory, a processor, and a computer program stored in the memory and operable on the processor, where the processor executes the computer program to implement the implementation of the present application.
  • FIG. 1 is a schematic flowchart diagram of a photo repairing method according to an embodiment of the present application
  • FIG. 2 is a schematic diagram of a scenario of a photo repairing method according to an embodiment of the present disclosure
  • FIG. 3 is a schematic flowchart of another method for repairing a photograph according to an embodiment of the present application
  • FIG. 4 is a schematic diagram of another scenario of a photo repairing method according to an embodiment of the present disclosure.
  • FIG. 5 is a schematic flowchart of another method for repairing a photograph according to an embodiment of the present application.
  • FIG. 6 is a schematic flowchart diagram of another method for repairing a photograph according to an embodiment of the present application.
  • FIG. 7 is a schematic flowchart diagram of another method for repairing a photograph according to an embodiment of the present application.
  • FIG. 8 is a schematic flowchart diagram of another method for repairing a photograph according to an embodiment of the present application.
  • FIG. 9 is a schematic diagram of initial three-dimensional data provided by an embodiment of the present application.
  • FIG. 10 is a schematic flowchart diagram of another method for repairing a photograph according to an embodiment of the present application.
  • FIG. 11 is a structural block diagram of a camera repairing apparatus according to an embodiment of the present application.
  • FIG. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • FIG. 13 is a schematic structural diagram of another terminal device according to an embodiment of the present disclosure.
  • Three-dimensional cameras are becoming more and more popular. More and more terminal devices use three-dimensional cameras. Compared with traditional cameras, three-dimensional cameras can collect more information. This application can optimize the terminal equipment according to the three-dimensional information collected by the three-dimensional camera. Photo technology.
  • FIG. 1 is a schematic flowchart of a photo repairing method according to an embodiment of the present disclosure.
  • the method may be implemented by a photo reparing device, where the device may be implemented by software and/or hardware, and may be integrated into a terminal device or integrated. In other devices with an operating system installed. As shown in FIG. 1, the method includes: step S110, step S111, and step S112.
  • step S110 when the recognition camera captures a defective object, the three-dimensional object data of the torn object is acquired by the recognition camera.
  • the missing object is an object that is partially blocked or missing.
  • the defective object is captured by the recognition camera, and the captured image includes a missing object.
  • the identification camera may be a camera on the terminal device, may be a front camera of the terminal device, and/or a rear camera.
  • at least one camera is generally provided in the related terminal device, and generally includes a front camera and a rear camera.
  • the recognition camera may be a front camera of the terminal device, so that the user can understand the captured picture through the screen of the terminal device.
  • the recognition camera may be a rear camera of the terminal device; the current user of the current terminal device may take other defects through the terminal device. object.
  • the image acquired by a conventional camera is generally two-dimensional data, that is, a color value (RGB value) or a set of gray values of pixels arranged in a matrix of rows and columns.
  • the three-dimensional data further includes the depth data of the captured missing object, that is, the distance between the different spatial points of the captured torn object and the camera, so the three-dimensional data can represent the spatial information of the captured object.
  • the recognition camera may be a camera with a distance sensor, and the distance sensor may acquire the distance between different spatial points on the captured object and the camera, so that the three-dimensional data of the photographed missing object may be acquired.
  • the three-dimensional data of the object includes: three-dimensional data of the missing object captured by the recognition camera.
  • the three-dimensional data of the object may be a set of three-dimensional data acquired when the missing object is stationary.
  • the three-dimensional data of the object may also be a plurality of sets of three-dimensional data acquired when the defective object is dynamic.
  • the torn object may be a partially occluded figure, which may be a face image that the user wears a hat and the visor obscures the eyes.
  • the three-dimensional data of the corresponding missing object includes three-dimensional data of other parts that are not occluded.
  • the recognition camera captures other objects, as shown in Fig. 2, it may be a damaged porcelain 113. In many museums, there are many porcelains with cultural relics and ornamental value, but partially damaged.
  • step S111 the three-dimensional data of the object is identified by a preset defect recognition model to determine a defective position on the defective object.
  • the three-dimensional data of the object is actually a set of data of one or more groups, and further needs to analyze and identify the three-dimensional data of the object according to the set of data, which actually corresponds to the position of the face that needs to be repaired.
  • the defect recognition model may be an identification system that has been trained to determine a defective position based on three-dimensional data of the object of the torn object.
  • the defect identification model may be pre-existing in the terminal device or pre-existing in the background server. When the three-dimensional data of the object needs to be recognized, the pre-stored defect recognition model is called to identify the three-dimensional data of the object to determine the defective position of the photographed missing object.
  • the incomplete position may be a position on the missing object in which the intrinsic feature is missing; if the incomplete object is a face in which the eye is blocked, the incomplete position is the position of the eye, and if the defective object is a broken object, the incomplete position It is the location where the damage occurred.
  • step S112 feature repair is performed on the incomplete position on the torn object.
  • the feature repairing process may be performed on the incomplete position of the three-dimensional data of the object according to the defect identification model.
  • the intrinsic feature corresponding to the missing can be determined according to the incomplete position, and the material of the corresponding intrinsic feature is added to the incomplete position.
  • the incomplete position is a face image in which the eye is occluded
  • the intrinsic feature of the missing missing position is the eye
  • the material of the corresponding eye feature is set at the incomplete position, so that the face image including the eye feature can be obtained.
  • the user can obtain a face image with complete features, which can increase the interest of the user to take pictures, and optimize the related photographing technology.
  • feature repairing the incomplete position on the torn object may be performed by the following steps S1121 and S1122.
  • step S1111 the missing information is determined according to the missing position, and the corresponding three-dimensional repair material is determined according to the missing information.
  • the missing information includes an intrinsic feature that is missing on the incomplete object, and characteristic information that the incomplete object itself has.
  • the three-dimensional repair material is a material file for filling in missing information that matches the missing information, and the three-dimensional repair material may be a system preset.
  • the torn object is a torn artifact of a museum
  • the three-dimensional restoration material may be a restoration material restored by the researcher according to historical materials.
  • the incomplete position is a hand part of a person image
  • the true intrinsic feature is a hand feature
  • the character feature information in the person image is a female
  • the missing information includes a female hand. It can be confirmed that there are many different places between the hand characteristics of the female and the hand characteristics of the male, and according to the characteristic information of the defective object, the more accurate three-dimensional repair material can be further obtained.
  • step S1122 feature repair is performed on the incomplete position according to the three-dimensional repair material.
  • the three-dimensional repair material may be placed at a position corresponding to the defective position, so that the missing information of the defective position in the defective object is filled, and an image of the complete object may be generated.
  • the missing object is porcelain 113
  • the defective position is the damaged area 114 occurring at the opening of the porcelain 113
  • the porcelain 113 is a green glazed porcelain
  • the missing information is determined according to the defective position.
  • the method includes: a damaged block at the opening of the porcelain, and a green glaze feature; and further determining, according to the missing information, a three-dimensional repair material 115 that matches the damaged area 114 and is a green glaze material, as shown in FIG.
  • the repair material 115 is placed at a position corresponding to the defective position 114, and an image of the complete porcelain 116 can be obtained.
  • Determining the missing information according to the missing position, and determining the corresponding three-dimensional repair material according to the missing information, can further improve the authenticity of the three-dimensional repair material.
  • a photographing and repairing solution provided in the embodiment of the present application is configured to acquire three-dimensional data of the object of the incomplete object by the identifying camera when the missing object is photographed by the identifying camera; and to obtain the three-dimensional object of the object by using a preset defect identifying model The data is identified to determine a location of the defect on the defective object; and a feature repair is performed on the defective location on the defective object.
  • FIG. 5 is a schematic flowchart of another method for repairing photographs according to an embodiment of the present invention.
  • an operation for performing feature repair on a defective position according to the three-dimensional repair material is optimized.
  • the method includes: step S120 to step S125.
  • step S120 when the recognition camera captures a defective object, the three-dimensional data of the object of the torn object is acquired by the recognition camera.
  • step S121 the three-dimensional data of the object is identified by a preset defect recognition model to determine a defective position on the defective object.
  • step S122 the missing information is determined according to the missing position, and the corresponding three-dimensional repair material is determined according to the missing information.
  • step S123 the photographed defective object is displayed in real time on the screen of the terminal device.
  • step S124 a display angle of the three-dimensional repair material is determined according to a photographing angle of the torn object, and a display position of the three-dimensional repair material is determined according to the incomplete position on the torn object.
  • step S125 the three-dimensional repair material is displayed on the screen of the terminal device according to the display angle and the display position.
  • the real-time display is to identify the image of the missing object acquired by the camera in real time. If the user adjusts the shooting angle of the recognition camera, the angle of the corresponding real-time displayed missing object also changes.
  • the display angle of the three-dimensional repair material is determined according to the photographing angle of the missing object, and the display angle of the three-dimensional repair material and the display angle of the missing object can be matched.
  • the display angle of the incomplete object will also affect the specific position of the incomplete position, and the three-dimensional repair material needs to match the incomplete position, so the display position of the three-dimensional repair material is determined according to the incomplete position, and the display position and the incomplete position of the three-dimensional repair material can be made. It is consistent with each other and can be consistent with the whole of the missing object.
  • the display angle may be a relative angle based on the missing object, and the display position may be a relative position based on the missing position.
  • the embodiment of the present application displays the missing object and the three-dimensional repair material in real time on the screen of the terminal device by determining the display angle and the display position of the three-dimensional repair material, and the three-dimensional repair material is also broken along with the user's adjustment of the shooting angle of the recognition camera.
  • the movement of the object in the screen moves together, and the user can see the complete defective object displayed in real time through the screen of the terminal device, so that the user's photographing operation is more interesting, and the photographing function of the terminal device is optimized.
  • FIG. 6 is a schematic flowchart of another method for photographing and repairing a method according to an embodiment of the present invention.
  • the operation of identifying the three-dimensional data of the object by using a preset defect identification model to determine a defective position on the disabled object is performed. Optimization, in an embodiment, as shown in FIG. 6, the method includes: step S130 to step S133.
  • step S130 when the recognition camera captures a defective object, the three-dimensional object data of the torn object is acquired by the recognition camera.
  • step S131 the three-dimensional data of the object is identified by a preset defect recognition model to determine an object type of the defective object.
  • step S132 a missing position on the torn object is determined according to the object type.
  • the number of defective objects may include various types, and may be, for example, a portrait of a photograph, a porcelain, or a sculpture.
  • Different types of objects have different inherent characteristics, so it is necessary to first determine the type of the object of the missing object, and then the position of the missing of the different objects can be determined according to the type of the object. After determining the type of the object of the defective object, the complete intrinsic feature of the object type can be further determined, and the inherent feature of the missing object can be determined.
  • step S133 feature repair is performed on the defective position on the torn object.
  • the feature repair of the defective position on the defective object can be implemented by:
  • step S1331 the missing information is determined according to the object type of the defective object and the defective position, and the corresponding three-dimensional repair material is determined according to the missing information.
  • step S1332 feature repair is performed on the incomplete position according to the three-dimensional repair material.
  • the missing information may be determined according to the type of the object and the missing position, thereby determining the three-dimensional repair material of the corresponding object type, and avoiding mismatching the materials of different shapes in different categories. On the disabled object, the accuracy of feature repair can be further improved.
  • the embodiment of the present application identifies the three-dimensional data of the object by using a preset defect identification model to determine an object type of the defective object; and determining a missing position on the disabled object according to the object type, which may be different according to the object.
  • the type to determine missing information can improve the accuracy of the identification of missing information.
  • FIG. 8 is a schematic flowchart of another method for repairing a photograph taken by an embodiment of the present invention, and an operation for acquiring three-dimensional data of an object of the torn object by the recognition camera is optimized.
  • the method includes: step S140 to step S144.
  • step S140 when the recognition camera captures the defective object, the position depth data of the torn object and the position infrared data are acquired by the recognition camera.
  • the recognition camera is a three-dimensional (3D, three Dimensional) camera, and the three-dimensional camera includes various hardware structures, and may include: an infrared sensor, a distance sensor, a lens, and the like.
  • the position depth data is a set of distance values of the spatial point distance recognition target included in the missing object; the position depth data of the missing object may be acquired by identifying a distance sensor in the camera.
  • the position infrared data is a collection of infrared data reflected by a spatial point included in the torn object.
  • the infrared sensor emits an infrared signal to the defective object, and the defective object reflects the infrared information, and the infrared sensor can realize the imaging of the defective object according to the received infrared data.
  • step S141 initial three-dimensional data of the torn object is determined based on the position depth data.
  • the position depth data includes the distance value of the spatial point included in the torn object, so the initial three-dimensional data of the torn object can be determined according to the position depth data.
  • points a, b, c, and d in the figure are four spatial points, and X, Y, and Z axes represent spaces, wherein the Z axis represents depth data of spatial points, and X and Y axes represent The position coordinate of the spatial point.
  • the depth data of point a is the largest, that is, the point a is the farthest distance from the recognition camera. It can be seen from the figure that a three-dimensional vertebral body can be formed according to the plane coordinates and depth data of the four spatial points, thereby The position depth data of the point and the plane coordinates of the spatial point can determine the initial three-dimensional data.
  • the corresponding detail position in the initial three-dimensional data may cause data loss, so it is further necessary to correct the initial three-dimensional data according to the position infrared data.
  • step S142 the initial three-dimensional data is corrected according to the position infrared data to obtain three-dimensional data of the object of the torn object.
  • the depth data of each spatial point corresponds to the infrared data one by one.
  • the corresponding infrared data can be used to measure and compare the initial three-dimensional data, and then the feature points of the missing spatial points are complemented.
  • the infrared signal is an electromagnetic wave, and the human eye cannot see the infrared signal. However, if the infrared light can still propagate at night or when the environment is dark and there is no visible light, in a dark environment, the infrared data can also be generated. Clear imaging; in turn, the initial 3D data can be corrected based on positional infrared data.
  • the fitting relationship function may be established according to the depth data of the adjacent points and the infrared data, and the corresponding depth data is calculated according to the fitting relationship function and the position infrared data of the missing spatial point, thereby obtaining the corrected object three-dimensional.
  • Data wherein the missing spatial point is a spatial point where the depth data is missing, and the adjacent spatial point is an adjacent spatial point of the missing spatial point.
  • step S143 the three-dimensional data of the object is identified by a preset defect recognition model to determine a defective position on the defective object.
  • step S144 feature repair is performed on the missing position on the torn object.
  • the image of the torn object that is, the acquired two-dimensional data of the incomplete object
  • the shape of the missing object can be recognized by the image processing recognition technique.
  • the two-dimensional data only includes the data of the planar image, and the requirements for the light are high. If the user photographs the defective object in a dark environment, the acquired image of the planar image may not be able to identify the accurate missing position, so the two-dimensional data The accuracy is lower.
  • the embodiment of the present application determines the initial three-dimensional data of the torn object according to the position depth data, and corrects the initial three-dimensional data according to the position infrared data to obtain three-dimensional data of the object of the torn object;
  • the dark position is recognized, and the initial three-dimensional data can be corrected by the position infrared data to obtain accurate three-dimensional data of the object, thereby improving the accuracy of the identification of the defective position.
  • FIG. 10 is a schematic flowchart of another method for repairing a photographing object according to an embodiment of the present invention.
  • an operation of acquiring three-dimensional data of an object of the incomplete object by using the recognizing camera Optimized in an embodiment, as shown in FIG. 10, the method includes: step S150 to step S153.
  • step S150 the preset sample data is input into a preset classifier for training to obtain a defect recognition model.
  • the defect identification model is configured to determine a corresponding incomplete position according to the three-dimensional data of the object of the captured torn object; the preset sample data includes sample three-dimensional data of the incomplete object, and a corresponding sample incomplete position.
  • the preset sample data may include a plurality of different sample data, and the different sample data corresponds to the acquired different missing objects, the corresponding sample three-dimensional data and the corresponding missing position; for example, if the defective object is For the character image, the preset sample data may be a sample three-dimensional data that blocks different person images of different parts and an annotation of the corresponding defective position.
  • the corresponding preset sample data is The sample includes three-dimensional data of the missing object, the type of the object of the defective object, and the corresponding missing position. Since each of the missing objects is different, and the type of the object is different, the corresponding missing position is also different, so the sample three-dimensional data, the corresponding object type, and the corresponding missing position are used as preset sample data input value preset classifiers.
  • the training is performed to obtain the defect recognition model, and the three-dimensional data of the input object can be identified to determine the corresponding object type and the corresponding missing position.
  • the preset classifier may be a neural network, and the preset sample data is input into a preset preset classifier for training; the preset classifier may extract feature data of the sample three-dimensional data, and the corresponding three-dimensional data is marked with the corresponding missing position. And/or object type, so the corresponding missing object incomplete position and/or object type can be determined based on the extracted feature data.
  • the obtained defect recognition model can determine the object type of different torn objects and determine the corresponding incomplete position.
  • step S151 when the recognition camera captures a defective object, the three-dimensional data of the object of the torn object is acquired by the recognition camera.
  • step S152 the three-dimensional data of the object is identified by a preset defect recognition model to determine a missing position on the disabled object.
  • step S153 feature repair is performed on the incomplete position on the torn object.
  • the obtained defect recognition model can determine the object type of different defective objects, and can improve the accuracy of the determination of the defective position.
  • FIG. 11 is a structural block diagram of a photo repairing apparatus according to an embodiment of the present disclosure.
  • the apparatus may perform a photographing repairing method. As shown in FIG. 11, the apparatus includes:
  • the three-dimensional data acquisition module 210 is configured to acquire, by the recognition camera, three-dimensional data of the object of the torn object when the recognition camera captures a defective object;
  • the location determining module 211 is configured to identify the three-dimensional data of the object by using a preset defect identification model to determine a missing position on the disabled object;
  • the repair processing module 212 is configured to perform feature repair on the incomplete position on the torn object.
  • a photographing and repairing solution provided in the embodiment of the present application is configured to acquire three-dimensional data of the object of the incomplete object by the identifying camera when the missing object is photographed by the identifying camera; and to obtain the three-dimensional object of the object by using a preset defect identifying model The data is identified to determine a location of the defect on the defective object; and a feature repair is performed on the defective location on the defective object.
  • the repair processing module may include, for example, a material determination module and a feature repair module.
  • a material determining module configured to determine missing information according to the missing position, and determine a corresponding three-dimensional repair material according to the missing information
  • the feature repair module is configured to perform feature repair on the incomplete position according to the three-dimensional repair material.
  • the feature repair module includes: a missing object display module, a display determination module, and a repair display module.
  • a defective object display module configured to display the photographed missing object in real time on a screen of the terminal device
  • a display determining module configured to determine a display angle of the three-dimensional repair material according to a shooting angle of the torn object, and determine a display position of the three-dimensional repair material according to the missing position on the torn object;
  • Fixing the display module configured to display the three-dimensional repair material on a screen of the terminal device according to the display angle and the display position.
  • the location determining module is further configured to:
  • a defective position on the torn object is determined according to the type of the object.
  • the repair processing module is further configured to:
  • Feature repair is performed on the incomplete position according to the three-dimensional repair material.
  • the recognition camera is a three-dimensional camera; the three-dimensional data acquisition module is further configured to:
  • the initial three-dimensional data is corrected according to the position infrared data to obtain three-dimensional data of the object of the torn object.
  • the method further includes: a training module.
  • the training module is configured to: before the three-dimensional data of the object is recognized by the preset defect recognition model, input the preset sample data into a preset classifier for training, to obtain a defect recognition model;
  • the defect identification model is configured to determine a corresponding incomplete position according to the three-dimensional data of the object of the captured torn object; the preset sample data includes sample three-dimensional data of the incomplete object, and a corresponding sample incomplete position.
  • a storage medium containing computer-executable instructions which is not limited to the photographing and repairing operation as described above, and may be related to the photographing repairing method provided by any embodiment of the present application. operating.
  • the embodiment of the present application further provides a storage medium including computer executable instructions, when executed by a computer processor, for performing a photo repairing method, the method comprising:
  • Feature repair is performed on the incomplete position on the missing object.
  • Storage media any of a variety of types of memory devices or storage devices.
  • the term "storage medium” is intended to include: a mounting medium such as a CD-ROM, a floppy disk or a tape device; a computer system memory or a random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.
  • Non-volatile memory such as flash memory, magnetic media (such as hard disk or optical storage); registers or other similar types of memory elements, and the like.
  • the storage medium may also include other types of memory or a combination thereof.
  • the storage medium may be located in a first computer system in which the program is executed, or may be located in a different second computer system, the second computer system being coupled to the first computer system via a network, such as the Internet.
  • the second computer system can provide program instructions to the first computer for execution.
  • the term "storage medium" can include two or more storage media that can reside in different locations (eg, in different computer systems connected through a network).
  • a storage medium may store program instructions (eg, embodied as a computer program) executable by one or more processors.
  • the embodiment of the present application provides a terminal device, where the camera repairing device provided by the embodiment of the present application can be integrated.
  • FIG. 12 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • the embodiment of the present application provides a terminal device 30, including a memory 31, a processor 32, and a computer stored in the memory 31 and operable on the processor. And a program, when the processor executes the computer program, implementing the photo repairing method described in the foregoing embodiment.
  • the terminal device provided by the embodiment of the present application can optimize related camera technologies.
  • FIG. 13 is a schematic structural diagram of a terminal device according to an embodiment of the present disclosure.
  • the terminal device may include: a casing (not shown), a touch screen (not shown), a touch button (not shown), a memory 301, and a central processing unit (Central Processing). Unit, CPU) 302 (also referred to as a processor, hereinafter referred to as CPU), a circuit board (not shown), and a power supply circuit (not shown).
  • CPU Central Processing Unit
  • the circuit board is disposed inside a space enclosed by the casing; the CPU 302 and the memory 301 are disposed on the circuit board; and the power circuit is configured to supply power to each circuit or device of the terminal device;
  • the memory 301 wherein executable program code is stored; the CPU 302 runs a computer program corresponding to the executable program code by reading executable program code stored in the memory 301 to implement the following steps:
  • Feature repair is performed on the incomplete position on the missing object.
  • the terminal device further includes: a peripheral interface 303, a radio frequency (RF) circuit 305, an audio circuit 306, a speaker 311, a power management chip 308, an input/output (I/O) subsystem 309, a touch screen 312, and others.
  • Input/control device 310 and external port 304 are communicated via one or more communication buses or signal lines 307.
  • the illustrated terminal device 300 is merely one example of a terminal device, and that the terminal device 300 may have more or fewer components than those shown in the figures, and two or more components may be combined. Or it can have different component configurations.
  • the various components shown in the figures can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.
  • the terminal device for implementing the photo-repairing provided in this embodiment is described in detail below.
  • the terminal device takes a mobile phone as an example.
  • the memory 301 can be accessed by the CPU 302, the peripheral interface 303, etc., and the memory 301 can include a high speed random access memory, and can also include a non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices.
  • a non-volatile memory such as one or more magnetic disk storage devices, flash memory devices. Or other volatile solid-state storage devices.
  • Peripheral interface 303 which can connect the input and output peripherals of the device to CPU 302 and memory 301.
  • I/O subsystem 309 which can connect input and output peripherals on the device, such as touch screen 312 and other input/control devices 310, to peripheral interface 303.
  • I/O subsystem 309 can include display controller 3091 and one or more input controllers 3092 for controlling other input/control devices 310.
  • one or more input controllers 3092 receive electrical signals from other input/control devices 310 or transmit electrical signals to other input/control devices 310, and other input/control devices 310 may include physical buttons (press buttons, rocker buttons, etc.) ), dial, slide switch, joystick, click wheel.
  • the input controller 3092 can be connected to any of the following: a keyboard, an infrared port, a USB interface, and a pointing device such as a mouse.
  • the touch screen 312 is an input interface and an output interface between the user terminal device and the user, and displays the visual output to the user.
  • the visual output may include graphics, text, icons, videos, and the like.
  • Display controller 3091 in I/O subsystem 309 receives an electrical signal from touch screen 312 or an electrical signal to touch screen 312.
  • the touch screen 312 detects the contact on the touch screen, and the display controller 3091 converts the detected contact into an interaction with the user interface object displayed on the touch screen 312, that is, realizes human-computer interaction, and the user interface object displayed on the touch screen 312 can be operated.
  • the device may also include a light mouse, which is a touch sensitive surface that does not display a visual output, or an extension of a touch sensitive surface formed by the touch screen.
  • the RF circuit 305 mainly establishes communication between the mobile phone and the wireless network (ie, the network side), and realizes data reception and transmission between the mobile phone and the wireless network. For example, sending and receiving short messages, emails, and the like. Specifically, the RF circuit 305 receives and transmits an RF signal, which is also referred to as an electromagnetic signal, and the RF circuit 305 converts the electrical signal into an electromagnetic signal or converts the electromagnetic signal into an electrical signal, and through the electromagnetic signal and communication network and other devices Communicate.
  • an RF signal which is also referred to as an electromagnetic signal
  • RF circuitry 305 may include known circuitry for performing these functions including, but not limited to, an antenna system, an RF transceiver, one or more amplifiers, a tuner, one or more oscillators, a digital signal processor, a codec CODER-DECoder (CODEC) chipset, Subscriber Identity Module (SIM), etc.
  • CDDEC codec CODER-DECoder
  • the audio circuit 306 mainly in which audio data is received from the peripheral interface 303, converts the audio data into an electrical signal, and transmits the electrical signal to the speaker 311.
  • the power management chip 308 provides power and power management for the hardware connected to the CPU 302, the I/O subsystem, and the peripheral interface.
  • the terminal device provided by the embodiment of the present application can optimize related camera technologies.
  • the photographing repairing device, the storage medium and the terminal device provided in the above embodiments can perform the photographing repairing method provided by any embodiment of the present application, and have the corresponding functional modules and beneficial effects of executing the method.
  • the photo repairing method provided by any embodiment of the present application.

Landscapes

  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
  • Processing Or Creating Images (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

公开了一种拍照修复方法以及装置、存储介质及终端设备，该方法通过在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；对所述残缺物体上的残缺位置进行特征修复。

Description

拍照修复方法以及装置、存储介质及终端设备
本申请要求在2018年05月16日提交中国专利局、申请号为201810468017.8的中国专利申请的优先权，该申请的全部内容通过引用结合在本申请中。
技术领域
本申请实施例涉及终端设备技术领域，例如涉及一种拍照修复方法以及装置、存储介质及终端设备。
背景技术
随着终端设备的拍照技术的发展，大部分的用户都拥有可以拍照的终端设备。通过终端设备进行拍照是非常容易的操作，用户只需要在对准拍摄物后按下拍照键就能完成拍照操作，可以以很方便地随时随地进行拍照。但是相关的拍照技术仍不完善，需要对相关的拍照技术进行优化。
发明内容
本申请实施例提供的一种拍照方法、装置、存储介质及终端设备，可以优化相关的拍照技术。
第一方面，本申请实施例提供了一种拍照修复方法，包括：
在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；
通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
对所述残缺物体上的残缺位置进行特征修复。
第二方面，本申请实施例提供了一种拍照修复装置，包括：
三维数据获取模块，设置为在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；
位置确定模块，设置为通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
修复处理模块，设置为对所述残缺物体上的残缺位置进行特征修复。
第三方面，本申请实施例提供了一种计算机可读存储介质，其上存储有计算机程序，该 程序被处理器执行时实现如本申请实施例所述的拍照修复方法。
第四方面，本申请实施例提供了一种终端设备，包括存储器，处理器及存储在存储器上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如本申请实施例所述的拍照修复方法。
附图概述
图1为本申请实施例提供的一种拍照修复方法的流程示意图；
图2为本申请实施例提供的一种拍照修复方法的场景示意图；
图3为本申请实施例提供的另一种拍照修复方法的流程示意图；
图4为本申请实施例提供的另一种拍照修复方法的场景示意图；
图5为本申请实施例提供的另一种拍照修复方法的流程示意图；
图6为本申请实施例提供的另一种拍照修复方法的流程示意图；
图7为本申请实施例提供的另一种拍照修复方法的流程示意图；
图8为本申请实施例提供的另一种拍照修复方法的流程示意图；
图9为本申请实施例提供的初始三维数据的示意图；
图10为本申请实施例提供的另一种拍照修复方法的流程示意图；
图11为本申请实施例提供的一种拍照修复装置的结构框图；
图12为本申请实施例提供的一种终端设备的结构示意图；
图13为本申请实施例提供的另一种终端设备的结构示意图。
具体实施方式
下面结合附图并通过具体实施方式来进一步说明本申请的技术方案。可以理解的是，此处所描述的具体实施例仅仅用于解释本申请，而非对本申请的限定。另外还需要说明的是，为了便于描述，附图中仅示出了与本申请相关的部分而非全部结构。
在更加详细地讨论示例性实施例之前应当提到的是，一些示例性实施例被描述成作为流程图描绘的处理或方法。虽然流程图将各步骤描述成顺序的处理，但是其中的许多步骤可以被并行地、并发地或者同时实施。此外，各步骤的顺序可以被重新安排。当其操作完成时所述处理可以被终止，但是还可以具有未包括在附图中的附加步骤。所述处理可以对应于方法、函数、规程、子例程、子程序等等。
三维摄像头逐渐普及开，越来越多的终端设备都采用了三维摄像头，三维摄像头相较于传统的摄像头可以采集到更多的信息，本申请可以根据三维摄像头采集的三维信息来优化终 端设备的拍照技术。
图1为本申请实施例提供的一种拍照修复方法的流程示意图，该方法可以由拍照修复装置执行，其中该装置可以由软件和/或硬件实现，一般可以集成在终端设备中，也可以集成在其他安装有操作系统的设备中。如图1所示，该方法包括：步骤S110、步骤S111和步骤S112。
在步骤S110中，在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据。
其中，所述残缺物体为部分部位被遮挡或缺损的物体。所述残缺物体为通过识别摄像头进行拍摄，所拍摄的画面中包括残缺物体。
所述识别摄像头可以是终端设备上的摄像头，可以是终端设备的前置摄像头，和/或后置摄像头。示例性地，相关的终端设备中一般都设置有至少一个摄像头，一般都包括前置摄像头和后置摄像头。如果拍摄的用户是终端设备的当前使用者，所述识别摄像头可以是终端设备的前置摄像头，以便用户可以通过终端设备的屏幕了解到拍摄的画面。再如，拍摄的对象不是终端设备的当前使用者，而是其他用户或者其他物体，则所述识别摄像头可以是终端设备的后置摄像头；当前终端设备的当前使用者通过终端设备来拍摄其他残缺物体。
传统的相机进行拍摄所获取的图像一般是二维数据，即以行列矩阵规则进行排列的像素点的色彩值(RGB值)或灰度值的集合。相比二维数据，所述三维数据中还包括拍摄到的残缺物体的深度数据，即拍摄的残缺物体的不同空间点与摄像头的距离，所以三维数据可以表示所拍摄的物体的空间信息。所述识别摄像头可以是带有距离传感器的摄像头，距离传感器可以获取所拍摄的物体上的不同空间点与摄像头的距离，如此可以获取到拍摄的残缺物体的三维数据。
所述物体三维数据包括：所述识别摄像头拍摄的所述残缺物体的三维数据。所述物体三维数据可以是残缺物体静止时所获取的一组三维数据。物体三维数据还可以是残缺物体为动态时所获取的多组三维数据。
示例性地，如果识别摄像头拍摄是人物像，则所述残缺物体可以是部分被遮挡的人物像，可以是用户带着帽子，帽檐将眼睛遮挡了的人脸像。则对应的残缺物体的物体三维数据包括其他未被遮挡的其他部位的三维数据。如果识别摄像头拍摄的是其他物体，如图2所示，可以是发生了破损的瓷器113，在诸多博物馆中都有很多具有文物研究价值和观赏价值，却局部破损的瓷器。
在步骤S111中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置。
所述物体三维数据实际上是一组或多组的数据的集合，进一步需要根据数据的集合来分 析识别该物体三维数据实际上对应的是脸部的需要修复的位置。
所述残缺识别模型可以是已经训练好的用于根据残缺物体的物体三维数据确定残缺位置的识别系统。所述残缺识别模型可以是预存在终端设备中，或预存在后台服务器中。在需要对物体三维数据进行识别时，调用预存的残缺识别模型来识别物体三维数据，以确定拍摄的残缺物体的残缺位置。
所述残缺位置可以是残缺物体上缺失了固有特征的位置；如果所述残缺物体是眼睛被遮挡的脸部，则残缺位置即眼睛的位置，如果所述残缺物体是破损的物体，则残缺位置是发生破损的位置。
在步骤S112中，对所述残缺物体上的残缺位置进行特征修复。
其中，可以根据残缺识别模型对所述物体三维数据的残缺位置进行特征修复处理。可以根据残缺位置确定对应缺失的固有特征，并在残缺位置上添加对应的固有特征的素材。示例性地，如果所述残缺位置为眼睛被遮挡的人脸像，则残缺位置缺失的固有特征是眼睛，在残缺位置上设置对应的眼睛特征的素材，以便可以得到包括眼睛特征的人脸图像；如此用户可以获取到具有完整特征的人脸图像，可以增加用户拍照的趣味性，优化相关的拍照技术。
在一实施例中，如图3所示，对所述残缺物体上的残缺位置进行特征修复可以通过下述步骤S1121和步骤S1122的方式实施。
在步骤S1121中，根据所述残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材。
其中，所述缺失信息包括残缺物体上缺失的固有特征，以及所述残缺物体本身具有的特征信息。所述三维修复素材为和缺失信息相吻合的用于填补缺失信息的素材文件，所述三维修复素材可以是系统预设。示例性地，如果所述残缺物体是博物馆的残缺文物，则三维修复素材可以是研究人员根据史料复原的修复素材。
示例性地，如果所述残缺位置为人物图像的手部部位，则确实的固有特征为手部特征，而如果所述人物图像中的人物特征信息为女性，则缺失信息包括女性的手部。可以确定的是，女性的手部特征和男性的手部特征有很多不同的地方，根据残缺物体本身具有的特征信息，可以进一步得到更准确的三维修复素材。
在步骤S1122中，根据所述三维修复素材对残缺位置进行特征修复。
其中，可以是将所述三维修复素材设置在和残缺位置对应的位置上，以使残缺物体中的残缺位置的残缺信息被填补，进而可以生成残缺物体完整的图像。
示例性地，如图2和图4所示，残缺物体为瓷器113，残缺位置为瓷器113的开口处发生的破损区域114，同时瓷器113为青釉瓷器，所以根据所述残缺位置确定缺失信息包括： 瓷器开口处的破损块，以及青釉特征；进而根据所述缺失信息确定了与破损区域114相吻合，以及是青釉材质的三维修复素材115，如图4所示，将所述三维修复素材115设置在于残缺位置114对应的位置上，可以得到完整的瓷器116的图像。
根据所述残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材，可以进一步提高三维修复素材的真实性。
本申请实施例中提供的一种拍照修复方案，通过在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；对所述残缺物体上的残缺位置进行特征修复。通过采用上述技术方案，可以根据残缺物体的物体三维数据对拍摄的残缺物体进行残缺修复，丰富终端设备与用户的交互功能，优化相关的拍照技术。
图5为本申请实施例提供的另一种拍照修复方法的流程示意图，在上述实施例所提供的技术方案的基础上，对根据所述三维修复素材对残缺位置进行特征修复的操作进行了优化，在一实施例中，如图5所示，该方法包括：步骤S120至步骤S125。
在步骤S120中，在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据。
在步骤S121中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置。
在步骤S122中，根据所述残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
在步骤S123中，在终端设备的屏幕上实时显示所拍摄到的残缺物体。
在步骤S124中，根据所述残缺物体的拍摄角度确定三维修复素材的显示角度，以及根据所述残缺物体上的残缺位置确定所述三维修复素材的显示位置。
在步骤S125中，根据所述显示角度和显示位置在终端设备的屏幕上显示所述三维修复素材。
其中，实时显示的为识别摄像头实时获取到的残缺物体的图像，如果用户调整识别摄像头的拍摄角度，相应的实时显示的残缺物体的角度也会发生变化。根据所述残缺物体的拍摄角度确定三维修复素材的显示角度，进而可以使三维修复素材的显示角度和残缺物体的显示角度吻合。
同时，残缺物体的显示角度也会影响残缺位置的具体位置，而三维修复素材需要与残缺位置相吻合，所以根据残缺位置确定三维修复素材的显示位置，可以使三维修复素材的显示 位置和残缺位置相吻合，进而可以和残缺物体的整体相吻合。
所述显示角度可以是以残缺物体为基准的相对角度，所述显示位置可以是以残缺位置为基准的相对位置。
本申请实施例通过确定三维修复素材的显示角度和显示位置，在终端设备的屏幕上实时显示残缺物体和三维修复素材，且随着用户调整识别摄像头的拍摄角度，三维修复素材也会随着残缺物体在画面中的移动一起移动，用户可以通过终端设备的屏幕看到实时显示的完整的残缺物体，使用户的拍照操作更有趣味性，优化终端设备的拍照功能。
图6为本申请实施例提供的另一种拍照修复方法的流程示意图，对通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置的操作进行了优化，在一实施例中，如图6所示，该方法包括：步骤S130至步骤S133。
在步骤S130中，在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据。
具体实施方式可以参考上文的相关描述，在此不再赘述。
在步骤S131中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型。
在步骤S132中，根据所述物体类型确定所述残缺物体上的残缺位置。
其中，残缺物体的可能包括多种类型，示例性地，可以是拍摄的人像，可以是瓷器，还可以是雕塑。而不同的类型的物体，其拥有的固有特征不相同，所以需要首先确定残缺物体的物体类型，进而可以根据物体的类型来对不同物体的残缺位置进行确定。在确定所述残缺物体的物体类型后，则可以进一步确定该物体类型的完整的固有特征，以及可以判断该残缺物体所缺失的固有特征是什么。
在步骤S133中，对所述残缺物体上的残缺位置进行特征修复。
具体实施方式可以参考上文的相关描述，在此不再赘述。
在一实施例中，如图7所示，对所述残缺物体上的残缺位置进行特征修复可以通过下述方式实施：
在步骤S1331中，根据所述残缺物体的物体类型以及所述残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材。
在步骤S1332中，根据所述三维修复素材对残缺位置进行特征修复。
在确定了残缺物体的物体类型后，可以根据物体类型以及所述残缺位置确定缺失信息确定缺失信息，进而确定对应的物体类型的三维修复素材，避免将形状接近的素材错用在不同的类别的残缺物体上，可以进一步提高特征修复的准确性。
本申请实施例通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型；根据所述物体类型确定所述残缺物体上的残缺位置，可以根据物体的不同类型来确定缺失信息，可以提高缺失信息的识别的准确性。
图8为本申请实施例提供的另一种拍照修复方法的流程示意图，对通过所述识别摄像头获取所述残缺物体的物体三维数据的操作进行了优化，在一实施例中，如图8所示，该方法包括：步骤S140至步骤S144。
在步骤S140中，在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的位置深度数据，以及位置红外数据。
所述识别摄像头为三维(3D,Three Dimensional)摄像头，三维摄像头中包括多种硬件结构，可包括：红外传感器、距离传感器和镜头等。
所述位置深度数据为残缺物体所包括的空间点距离识别摄像头的距离值的集合；可以通过识别摄像头中的距离传感器获取残缺物体的位置深度数据。
所述位置红外数据为残缺物体所包括的空间点反射的红外数据的集合。其中，三维摄像头在拍摄时，其中红外传感器发射红外信号至残缺物体，残缺物体会对红外信息进行反射，红外传感器根据接收到的反射的红外数据可以实现残缺物体的成像。
在步骤S141中，根据所述位置深度数据确定所述残缺物体的初始三维数据。
其中，位置深度数据中包括了残缺物体所包括的空间点的距离值，所以可以根据位置深度数据确定残缺物体的初始三维数据。示例性地，如图9所示，图中点a、b、c和d为四个空间点，X、Y和Z轴表示空间，其中Z轴表示空间点的深度数据，X和Y轴表示空间点的平面位置坐标。其中点a的深度数据最大，也就是点a距离识别摄像头的距离最远，从图中可以看到根据四个空间点的平面坐标和深度数据可以形成一个三维的椎体，从而根据多个空间点的位置深度数据以及空间点的平面坐标可以确定初始的三维数据。
但是如果残缺物体的某些细节处被遮挡或者发生数据丢失的情况，则初始三维数据中对应的细节位置会出现数据缺失的问题，所以进一步需要根据位置红外数据对初始三维数据进行校正。
在步骤S142中，根据所述位置红外数据对所述初始三维数据进行校正，以得到所述残缺物体的物体三维数据。
其中，对于残缺物体所包括的空间点，每个空间点的深度数据和红外数据一一对应。对于数据缺失的空间点的深度数据，根据其对应的红外数据可以通过对整体的初始三维数据进行衡量和比对，进而对缺失的空间点进行特征补全。红外信号是一种电磁波，人眼无法看到红外信号，但是如果在夜晚或者环境较暗没有可见光的时候，红外光依然可以进行传播，所 以在较暗的环境中，根据红外数据也可以生成较清晰的成像；进而可以根据位置红外数据来对初始三维数据进行校正。
在一实施例中，可以根据相邻点的深度数据和红外数据建立拟合关系函数，并根据拟合关系函数以及缺失空间点的位置红外数据计算对应的深度数据，进而得到校正后的物体三维数据；其中，缺失空间点为深度数据缺失的空间点，相邻空间点为缺失空间点的相邻的空间点。
在步骤S143中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置。
在步骤S144中，对所述残缺物体上的残缺位置进行特征修复。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
虽然通过普通摄像头来拍摄残缺物体的图像，即获取的残缺物体的二维数据，通过图像处理识别技术也可以识别残缺物体的形状。但是二维数据仅包括平面图像的数据，对于光线的要求较高，如果用户在较暗的环境中拍摄残缺物体，所获取的平面图像的数据中可能无法识别准确的残缺位置，所以二维数据的准确性较低。
本申请实施例通过根据所述位置深度数据确定所述残缺物体的初始三维数据，根据所述位置红外数据对所述初始三维数据进行校正，以得到所述残缺物体的物体三维数据；在光线较暗的位置进行识别，也能通过位置红外数据对初始三维数据进行校正，得到准确的物体三维数据，进而可以提高残缺位置的识别的精确性。
图10为本申请实施例提供的另一种拍照修复方法的流程示意图，在上述任意实施例所提供的技术方案的基础上，对通过所述识别摄像头获取所述残缺物体的物体三维数据的操作进行了优化，在一实施例中，如图10所示，该方法包括：步骤S150至步骤S153。
在步骤S150中，将预设样本数据输入至预设分类器中进行训练，得到残缺识别模型。
其中，所述残缺识别模型用于根据拍摄到的残缺物体的物体三维数据确定对应的残缺位置；所述预设样本数据包括残缺物体的样本三维数据，以及对应的样本残缺位置。
其中，所述预设样本数据可以包括多个不同的样本数据，不同的样本数据对应为获取的不同的残缺物体，所对应的样本三维数据和对应的残缺位置；示例性地，如果残缺物体是人物像，预设样本数据可以是遮挡不同部位的不同的人物像的样本三维数据以及对应的残缺位置的标注。
如果通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型，以及根据所述物体类型确定所述残缺物体上的残缺位置，则对应的预设样本数据包括残缺物体的样本三维数据、残缺物体的物体类型以及对应的残缺位置。由于每个残缺物体 有所不同，以及物体类型有所不同，对应的残缺位置也有所不同，所以将样本三维数据、对应的物体类型以及对应的残缺位置作为预设样本数据输入值预设分类器中进行训练，以得到残缺识别模型，可以根据输入的物体三维数据进行识别，确定对应的物体类型，以及对应的残缺位置。
所述预设分类器可以是神经网络，通过将预设样本数据输入值预设分类器中进行训练；预设分类器可以提取样本三维数据的特征数据，由于样本三维数据标注了对应的残缺位置和/或物体类型，所以可以根据提取的特征数据确定对应的残缺物体残缺位置和/或物体类型。
通过预设样本数据对预设分类器进行训练后，得到的残缺识别模型可以对不同的残缺物体的物体类型进行确定，以及确定对应的残缺位置。
在步骤S151中，在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据。
在步骤S152中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置。
在步骤S153中，对所述残缺物体上的残缺位置进行特征修复。
上述操作的具体实施方式可以参考上文的相关描述，在此不再赘述。
通过预设样本数据对预设分类器进行训练后，得到的残缺识别模型可以对不同的残缺物体的物体类型进行确定，可以提高残缺位置的确定的准确性。
图11为本申请实施例提供的一种拍照修复装置的结构框图，该装置可以执行拍照修复方法，如图11所示，该装置包括：
三维数据获取模块210，设置为在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；
位置确定模块211，设置为通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
修复处理模块212，设置为对所述残缺物体上的残缺位置进行特征修复。
本申请实施例中提供的一种拍照修复方案，通过在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；对所述残缺物体上的残缺位置进行特征修复。通过采用上述技术方案，可以根据残缺物体的物体三维数据对拍摄的残缺物体进行残缺修复，丰富终端设备与用户的交互功能，优化相关的拍照技术。
在一实施例中，修复处理模块例如可以包括：素材确定模块和特征修复模块。
素材确定模块，设置为根据所述残缺位置确定缺失信息，根据所述缺失信息确定对应的 三维修复素材；
特征修复模块，设置为根据所述三维修复素材对残缺位置进行特征修复。
在一实施例中，特征修复模块包括：残缺物体显示模块、显示确定模块和修复显示模块。
残缺物体显示模块，设置为在终端设备的屏幕上实时显示所拍摄到的残缺物体；
显示确定模块，设置为根据所述残缺物体的拍摄角度确定三维修复素材的显示角度，以及根据所述残缺物体上的残缺位置确定所述三维修复素材的显示位置；
修复显示模块，设置为根据所述显示角度和显示位置在终端设备的屏幕上显示所述三维修复素材。
在一实施例中，位置确定模块还设置为：
通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型；
根据所述物体类型确定所述残缺物体上的残缺位置。
在一实施例中，修复处理模块还设置为：
根据所述残缺物体的物体类型以及所述残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材；
根据所述三维修复素材对残缺位置进行特征修复。
在一实施例中，所述识别摄像头为三维摄像头；三维数据获取模块还设置为：
通过所述识别摄像头获取所述残缺物体的位置深度数据，以及位置红外数据；
根据所述位置深度数据确定所述残缺物体的初始三维数据；
根据所述位置红外数据对所述初始三维数据进行校正，以得到所述残缺物体的物体三维数据。
在一实施例中，还包括：训练模块。
训练模块，设置为在通过预设的残缺识别模型对所述物体三维数据进行识别之前，将预设样本数据输入至预设分类器中进行训练，得到残缺识别模型；
其中，所述残缺识别模型用于根据拍摄到的残缺物体的物体三维数据确定对应的残缺位置；所述预设样本数据包括残缺物体的样本三维数据，以及对应的样本残缺位置。
本申请实施例所提供的一种包含计算机可执行指令的存储介质，其计算机可执行指令不限于如上所述的拍照修复操作，还可以执行本申请任意实施例所提供的拍照修复方法中的相关操作。
本申请实施例还提供一种包含计算机可执行指令的存储介质，所述计算机可执行指令在由计算机处理器执行时用于执行拍照修复方法，该方法包括：
在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；
通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
对所述残缺物体上的残缺位置进行特征修复。
存储介质——任何的各种类型的存储器设备或存储设备。术语“存储介质”旨在包括：安装介质，例如CD-ROM、软盘或磁带装置；计算机系统存储器或随机存取存储器，诸如DRAM、DDR RAM、SRAM、EDO RAM，兰巴斯(Rambus)RAM等；非易失性存储器，诸如闪存、磁介质(例如硬盘或光存储)；寄存器或其它相似类型的存储器元件等。存储介质可以还包括其它类型的存储器或其组合。另外，存储介质可以位于程序在其中被执行的第一计算机系统中，或者可以位于不同的第二计算机系统中，第二计算机系统通过网络(诸如因特网)连接到第一计算机系统。第二计算机系统可以提供程序指令给第一计算机用于执行。术语“存储介质”可以包括可以驻留在不同位置中(例如在通过网络连接的不同计算机系统中)的两个或更多存储介质。存储介质可以存储可由一个或多个处理器执行的程序指令(例如具体实现为计算机程序)。
本申请实施例提供了一种终端设备，该终端设备中可集成本申请实施例提供的拍照修复装置。
图12为本申请实施例提供的一种终端设备的结构示意图，本申请实施例提供了一种终端设备30，包括存储器31，处理器32及存储在存储器31上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现上述实施例所述的拍照修复方法。本申请实施例提供的终端设备，可以优化相关的拍照技术。
图13为本申请实施例提供的一种终端设备的结构示意图。如图13所示，该终端设备可以包括：壳体(图中未示出)、触摸屏(图中未示出)、触摸按键(图中未示出)、存储器301、中央处理器(Central Processing Unit，CPU)302(又称处理器，以下简称CPU)、电路板(图中未示出)和电源电路(图中未示出)。所述电路板安置在所述壳体围成的空间内部；所述CPU302和所述存储器301设置在所述电路板上；所述电源电路，其中为所述终端设备的各个电路或器件供电；所述存储器301，其中存储可执行程序代码；所述CPU302通过读取所述存储器301中存储的可执行程序代码来运行与所述可执行程序代码对应的计算机程序，以实现以下步骤：
在识别摄像头拍摄到残缺物体时，通过所述识别摄像头获取所述残缺物体的物体三维数据；
通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
对所述残缺物体上的残缺位置进行特征修复。
所述终端设备还包括：外设接口303、射频(Radio Frequency，RF)电路305、音频电路306、扬声器311、电源管理芯片308、输入/输出(I/O)子系统309、触摸屏312、其他输入/控制设备310以及外部端口304，这些部件通过一个或多个通信总线或信号线307来通信。
应该理解的是，图示终端设备300仅仅是终端设备的一个范例，并且终端设备300可以具有比图中所示出的更多的或者更少的部件，可以组合两个或更多的部件，或者可以具有不同的部件配置。图中所示出的各种部件可以在包括一个或多个信号处理和/或专用集成电路在内的硬件、软件、或硬件和软件的组合中实现。
下面就本实施例提供的用于实现拍照修复的终端设备进行详细的描述，该终端设备以手机为例。
存储器301，所述存储器301可以被CPU302、外设接口303等访问，所述存储器301可以包括高速随机存取存储器，还可以包括非易失性存储器，例如一个或多个磁盘存储器件、闪存器件、或其他易失性固态存储器件。
外设接口303，所述外设接口303可以将设备的输入和输出外设连接到CPU302和存储器301。
I/O子系统309，所述I/O子系统309可以将设备上的输入输出外设，例如触摸屏312和其他输入/控制设备310，连接到外设接口303。I/O子系统309可以包括显示控制器3091和用于控制其他输入/控制设备310的一个或多个输入控制器3092。其中，一个或多个输入控制器3092从其他输入/控制设备310接收电信号或者向其他输入/控制设备310发送电信号，其他输入/控制设备310可以包括物理按钮(按压按钮、摇臂按钮等)、拨号盘、滑动开关、操纵杆、点击滚轮。值得说明的是，输入控制器3092可以与以下任一个连接：键盘、红外端口、USB接口以及诸如鼠标的指示设备。
触摸屏312，所述触摸屏312是用户终端设备与用户之间的输入接口和输出接口，将可视输出显示给用户，可视输出可以包括图形、文本、图标、视频等。
I/O子系统309中的显示控制器3091从触摸屏312接收电信号或者向触摸屏312发送电信号。触摸屏312检测触摸屏上的接触，显示控制器3091将检测到的接触转换为与显示在触摸屏312上的用户界面对象的交互，即实现人机交互，显示在触摸屏312上的用户界面对象可以是运行游戏的图标、联网到相应网络的图标等。值得说明的是，设备还可以包括光鼠，光鼠是不显示可视输出的触摸敏感表面，或者是由触摸屏形成的触摸敏感表面的延伸。
RF电路305，主要其中建立手机与无线网络(即网络侧)的通信，实现手机与无线网络的数据接收和发送。例如收发短信息、电子邮件等。具体地，RF电路305接收并发送RF信号，RF信号也称为电磁信号，RF电路305将电信号转换为电磁信号或将电磁信号转换为电信号，并且通过该电磁信号与通信网络以及其他设备进行通信。RF电路305可以包括用于执行这些功能的已知电路，其包括但不限于天线系统、RF收发机、一个或多个放大器、调谐器、一个或多个振荡器、数字信号处理器、编译码器(COder-DECoder，CODEC)芯片组、用户标识模块(Subscriber Identity Module，SIM)等等。
音频电路306，主要其中从外设接口303接收音频数据，将该音频数据转换为电信号，并且将该电信号发送给扬声器311。
扬声器311，其中将手机通过RF电路305从无线网络接收的语音信号，还原为声音并向用户播放该声音。
电源管理芯片308，其中为CPU302、I/O子系统及外设接口所连接的硬件进行供电及电源管理。
本申请实施例提供的终端设备，可以优化相关的拍照技术。
上述实施例中提供的拍照修复装置、存储介质及终端设备可执行本申请任意实施例所提供的拍照修复方法，具备执行该方法相应的功能模块和有益效果。未在上述实施例中详尽描述的技术细节，可参见本申请任意实施例所提供的拍照修复方法。

Claims (20)

  1. 一种拍照修复方法，包括：
    在识别摄像头拍摄到残缺物体的情况下，通过所述识别摄像头获取所述残缺物体的物体三维数据；
    通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
    对所述残缺物体上的残缺位置进行特征修复。
  2. 如权利要求1所述的方法，其中，对所述残缺物体上的残缺位置进行特征修复包括：
    根据所述残缺物体上的残缺位置确定缺失信息，根据所述缺失信息确定所述残缺物体对应的三维修复素材；
    根据所述三维修复素材对所述残缺物体上的残缺位置进行特征修复。
  3. 如权利要求2所述的方法，其中，根据所述三维修复素材对所述残缺物体上的残缺位置进行特征修复包括：
    在终端设备的屏幕上实时显示所拍摄到的残缺物体；
    根据所述残缺物体的拍摄角度确定三维修复素材的显示角度，以及根据所述残缺物体上的残缺位置确定所述三维修复素材的显示位置；
    根据所述三维修复素材的显示角度和所述三维修复素材的显示位置在终端设备的屏幕上显示所述三维修复素材。
  4. 如权利要求1所述的方法，其中，通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置包括：
    通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型；
    根据所述残缺物体的物体类型确定所述残缺物体上的残缺位置。
  5. 如权利要求4所述的方法，其中，对所述残缺物体上的残缺位置进行特征修复包括：
    根据所述残缺物体的物体类型以及所述残缺物体上的残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材；
    根据所述三维修复素材对残缺位置进行特征修复。
  6. 如权利要求1至5任一项所述的方法，其中，所述识别摄像头为三维摄像头；通过所述识别摄像头获取所述残缺物体的物体三维数据包括：
    通过所述识别摄像头获取所述残缺物体的位置深度数据，以及位置红外数据；
    根据所述残缺物体的位置深度数据确定所述残缺物体的初始三维数据；
    根据所述位置红外数据对所述初始三维数据进行校正，以得到所述残缺物体的物体三维 数据。
  7. 如权利要求6所述的方法，其中，所述位置深度数据为所述残缺物体所包括的空间点距离识别摄像头的距离值的集合；所述位置红外数据为所述残缺物体所包括的空间点反射的红外数据的集合。
  8. 如权利要求6所述的方法，其中，所述三维摄像头包括：红外传感器、距离传感器和镜头。
  9. 如权利要求1至8任一项所述的方法，在通过预设的残缺识别模型对所述物体三维数据进行识别之前，还包括：
    将预设样本数据输入至预设分类器中进行训练，以确定残缺识别模型；
    其中，所述残缺识别模型用于根据拍摄到的残缺物体的物体三维数据确定所述残缺物体对应的残缺位置；
    所述预设样本数据包括残缺物体的样本三维数据，以及所述残缺物体对应的样本残缺位置。
  10. 一种拍照修复装置，包括：
    三维数据获取模块，设置为在识别摄像头拍摄到残缺物体的情况下，通过所述识别摄像头获取所述残缺物体的物体三维数据；
    位置确定模块，设置为通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体上的残缺位置；
    修复处理模块，设置为对所述残缺物体上的残缺位置进行特征修复。
  11. 如权利要求10所述的装置，其中，所述修复处理模块包括：
    素材确定模块，设置为根据所述残缺物体上的残缺位置确定缺失信息，根据所述缺失信息确定所述残缺物体对应的三维修复素材；
    特征修复模块，设置为根据所述三维修复素材对所述残缺物体上的残缺位置进行特征修复。
  12. 如权利要求11所述的装置，其中，特征修复模块包括：
    残缺物体显示模块，设置为在终端设备的屏幕上实时显示所拍摄到的残缺物体；
    显示确定模块，设置为根据所述残缺物体的拍摄角度确定三维修复素材的显示角度，以及根据所述残缺物体上的残缺位置确定所述三维修复素材的显示位置；
    修复显示模块，设置为根据所述三维修复素材的显示角度和所述三维修复素材的显示位置在终端设备的屏幕上显示所述三维修复素材。
  13. 如权利要求10所述的装置，其中，所述位置确定模块还设置为：
    通过预设的残缺识别模型对所述物体三维数据进行识别，以确定所述残缺物体的物体类型；
    根据所述残缺物体的物体类型确定所述残缺物体上的残缺位置。
  14. 如权利要求13所述的装置，其中，所述修复显示模块还设置为：
    根据所述残缺物体的物体类型以及所述残缺物体上的残缺位置确定缺失信息，根据所述缺失信息确定对应的三维修复素材；
    根据所述三维修复素材对残缺位置进行特征修复。
  15. 如权利要求10-14任一项所述的装置，其中，所述识别摄像头为三维摄像头，所述三维数据获取模块还设置为：
    通过所述识别摄像头获取所述残缺物体的位置深度数据，以及位置红外数据；
    根据所述残缺物体的位置深度数据确定所述残缺物体的初始三维数据；
    根据所述位置红外数据对所述初始三维数据进行校正，以得到所述残缺物体的物体三维数据。
  16. 如权利要求15所述的装置，其中，所述位置深度数据为所述残缺物体所包括的空间点距离识别摄像头的距离值的集合；所述位置红外数据为所述残缺物体所包括的空间点反射的红外数据的集合。
  17. 如权利要求15所述的装置，其中，所述三维摄像头包括：红外传感器、距离传感器和镜头。
  18. 如权利要求10至17任一项所述的装置，还包括：
    训练模块，设置为将预设样本数据输入至预设分类器中进行训练，以确定残缺识别模型；
    其中，所述残缺识别模型用于根据拍摄到的残缺物体的物体三维数据确定所述残缺物体对应的残缺位置；所述预设样本数据包括残缺物体的样本三维数据，以及所述残缺物体对应的样本残缺位置。
  19. 一种计算机可读存储介质，其上存储有计算机程序，该程序被处理器执行时实现如权利要求1-9中任一项所述的拍照修复方法。
  20. 一种终端设备，包括存储器，处理器及存储在存储器上并可在处理器运行的计算机程序，所述处理器执行所述计算机程序时实现如权利要求1-9任一项所述的拍照修复方法。
PCT/CN2019/085452 2018-05-16 2019-05-05 拍照修复方法以及装置、存储介质及终端设备 WO2019218878A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810468017.8 2018-05-16
CN201810468017.8A CN108765321B (zh) 2018-05-16 2018-05-16 拍照修复方法、装置、存储介质及终端设备

Publications (1)

Publication Number Publication Date
WO2019218878A1 true WO2019218878A1 (zh) 2019-11-21

Family

ID=64008166

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/085452 WO2019218878A1 (zh) 2018-05-16 2019-05-05 拍照修复方法以及装置、存储介质及终端设备

Country Status (2)

Country Link
CN (1) CN108765321B (zh)
WO (1) WO2019218878A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108765321B (zh) * 2018-05-16 2021-09-07 Oppo广东移动通信有限公司 拍照修复方法、装置、存储介质及终端设备
CN112785633A (zh) * 2020-12-30 2021-05-11 深兰人工智能芯片研究院（江苏）有限公司 文物修复策略的获取方法、装置、电子设备及存储介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2317470A1 (en) * 2009-10-29 2011-05-04 Samsung Electronics Co., Ltd. Image inpainting apparatus and method using restricted search region
CN105763812A (zh) * 2016-03-31 2016-07-13 北京小米移动软件有限公司 智能拍照方法及装置
CN106600701A (zh) * 2016-10-28 2017-04-26 黑龙江省科学院自动化研究所 一种基于虚拟现实技术的数字博物馆文物展示方法
CN107343148A (zh) * 2017-07-31 2017-11-10 广东欧珀移动通信有限公司 图像补全方法、装置和终端
CN107392873A (zh) * 2017-07-28 2017-11-24 上海鋆创信息技术有限公司 物体虚拟修复方法、物体修复方法及物体虚拟展示系统
CN107437268A (zh) * 2017-07-31 2017-12-05 广东欧珀移动通信有限公司 拍照方法、装置、移动终端和计算机存储介质
CN107948499A (zh) * 2017-10-31 2018-04-20 维沃移动通信有限公司 一种图像拍摄方法及移动终端
CN108765321A (zh) * 2018-05-16 2018-11-06 Oppo广东移动通信有限公司 拍照修复方法、装置、存储介质及终端设备

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI419078B (zh) * 2011-03-25 2013-12-11 Univ Chung Hua 即時立體影像產生裝置與方法
CN105069833A (zh) * 2015-07-29 2015-11-18 苏州华漫信息服务有限公司 能够保持纹理信息数据的三维模型修补方法
CN105598450B (zh) * 2016-02-02 2017-11-10 陕西天元智能再制造股份有限公司 一种零部件损伤的激光立体仿形修复方法
CN106127147B (zh) * 2016-06-23 2019-07-26 深圳市唯特视科技有限公司 一种基于三维数据的人脸深度纹理修复方法
CN107452034B (zh) * 2017-07-31 2020-06-05 Oppo广东移动通信有限公司 图像处理方法及其装置

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP2317470A1 (en) * 2009-10-29 2011-05-04 Samsung Electronics Co., Ltd. Image inpainting apparatus and method using restricted search region
CN105763812A (zh) * 2016-03-31 2016-07-13 北京小米移动软件有限公司 智能拍照方法及装置
CN106600701A (zh) * 2016-10-28 2017-04-26 黑龙江省科学院自动化研究所 一种基于虚拟现实技术的数字博物馆文物展示方法
CN107392873A (zh) * 2017-07-28 2017-11-24 上海鋆创信息技术有限公司 物体虚拟修复方法、物体修复方法及物体虚拟展示系统
CN107343148A (zh) * 2017-07-31 2017-11-10 广东欧珀移动通信有限公司 图像补全方法、装置和终端
CN107437268A (zh) * 2017-07-31 2017-12-05 广东欧珀移动通信有限公司 拍照方法、装置、移动终端和计算机存储介质
CN107948499A (zh) * 2017-10-31 2018-04-20 维沃移动通信有限公司 一种图像拍摄方法及移动终端
CN108765321A (zh) * 2018-05-16 2018-11-06 Oppo广东移动通信有限公司 拍照修复方法、装置、存储介质及终端设备

Also Published As

Publication number Publication date
CN108765321B (zh) 2021-09-07
CN108765321A (zh) 2018-11-06

Similar Documents

Publication Publication Date Title
CN108764091B (zh) 活体检测方法及装置、电子设备和存储介质
US11481923B2 (en) Relocalization method and apparatus in camera pose tracking process, device, and storage medium
US10043308B2 (en) Image processing method and apparatus for three-dimensional reconstruction
WO2019101021A1 (zh) 图像识别方法、装置及电子设备
US20200265234A1 (en) Electronic device for providing shooting mode based on virtual character and operation method thereof
KR101979669B1 (ko) 이미지 내 사용자의 시선 보정 방법, 기계로 읽을 수 있는 저장 매체 및 통신 단말
US9210404B2 (en) Calibration and registration of camera arrays using a single circular grid optical target
WO2020103526A1 (zh) 拍照方法、装置、存储介质及终端设备
US11776307B2 (en) Arrangement for generating head related transfer function filters
WO2018140107A1 (en) System for 3d image filtering
CN110412765B (zh) 增强现实图像拍摄方法、装置、存储介质及增强现实设备
US20220309836A1 (en) Ai-based face recognition method and apparatus, device, and medium
CN109522863B (zh) 耳部关键点检测方法、装置及存储介质
CN113280752B (zh) 一种凹槽深度测量方法、装置、系统及激光测量设备
US11527014B2 (en) Methods and systems for calibrating surface data capture devices
WO2019100407A1 (zh) 基于图样中标志图形点坐标的转换关系定位终端屏幕
US20140354784A1 (en) Shooting method for three dimensional modeling and electronic device supporting the same
WO2022160587A1 (zh) 深度检测方法、装置、电子设备、存储介质及程序产品
CN108491780B (zh) 图像美化处理方法、装置、存储介质及终端设备
WO2019218878A1 (zh) 拍照修复方法以及装置、存储介质及终端设备
WO2019218879A1 (zh) 拍照交互方法、装置、存储介质及终端设备
CN112351188A (zh) 根据对象显示图形元素的装置和方法
US20180098050A1 (en) Power efficient long range depth sensing
CN113870213A (zh) 图像显示方法、装置、存储介质以及电子设备
CN110163192B (zh) 字符识别方法、装置及可读介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19803489

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19803489

Country of ref document: EP

Kind code of ref document: A1