WO2019237744A1 - 构建图像深度信息的方法及装置 - Google Patents

构建图像深度信息的方法及装置 Download PDF

Info

Publication number
WO2019237744A1
WO2019237744A1 PCT/CN2019/073070 CN2019073070W WO2019237744A1 WO 2019237744 A1 WO2019237744 A1 WO 2019237744A1 CN 2019073070 W CN2019073070 W CN 2019073070W WO 2019237744 A1 WO2019237744 A1 WO 2019237744A1
Authority
WO
WIPO (PCT)
Prior art keywords
depth information
added
key points
type
area
Prior art date
Application number
PCT/CN2019/073070
Other languages
English (en)
French (fr)
Inventor
赖锦锋
庄幽文
Original Assignee
北京微播视界科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京微播视界科技有限公司 filed Critical 北京微播视界科技有限公司
Publication of WO2019237744A1 publication Critical patent/WO2019237744A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/10—Processing, recording or transmission of stereoscopic or multi-view image signals
    • H04N13/106—Processing image signals
    • H04N13/128—Adjusting depth or disparity
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
    • H04N13/20—Image signal generators
    • H04N13/271—Image signal generators wherein the generated image signals comprise depth maps or disparity maps

Definitions

  • the present disclosure relates to the field of image processing, and in particular, to a method and device for constructing image depth information.
  • embodiments of the present disclosure provide a method and device for constructing image depth information, which at least partially solve the problems existing in the prior art.
  • an embodiment of the present disclosure provides a method for constructing image depth information, including:
  • the region where the target object is pre-added depth information includes:
  • a region where pre-added depth information is marked in the foreground.
  • the step of marking the pre-added depth information in the foreground information includes:
  • the marking a region with pre-added depth information in the foreground includes:
  • a region in which the depth information is pre-added is marked within the contour information.
  • the method further includes:
  • the contour information is smoothed.
  • the obtaining the depth information related to the target object is specifically:
  • the adding the acquired depth information to a pre-added depth information area includes:
  • the depth information in the depth information template is added to the pre-added depth information area.
  • adding the depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points includes:
  • the at least one second region is correspondingly attached to the at least one first region.
  • adding the depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points includes:
  • an embodiment of the present disclosure further provides an apparatus for constructing image depth information, including:
  • An acquisition module for acquiring depth information related to the target object
  • Area labeling module which is used to mark the target object with pre-added depth information
  • An adding module is used to add the acquired depth information to the pre-added depth information area.
  • the area marking module includes:
  • Separation module for separating the foreground and background of the target object
  • Extraction module for extracting the foreground of the target object
  • Foreground labeling module used for labeling a region with pre-added depth information in the foreground.
  • the foreground mark module includes:
  • Key point extraction module for extracting key points in the foreground
  • Area division module used for area division of the foreground based on the key points
  • Keypoint Marking Module Used to mark the keypoints of the pre-added depth information.
  • the foreground marking module includes:
  • a contour extraction module configured to extract contour information of the foreground
  • Contour marking module It is used to mark a region with pre-added depth information in the contour information.
  • Smoothing processing module used for smoothing the contour information extracted by the contour extraction module.
  • the obtaining module is configured to obtain depth information related to a target object, specifically:
  • the adding module includes:
  • the first type of key point extraction module used to extract the key points of the pre-added depth information, which are the first type of key points;
  • the second type of key point extraction module used to extract the key points of the depth information template, which are the second type of key points;
  • Information adding module It is used to add the depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points.
  • the information adding module includes:
  • a first segmentation module used for triangulating the pre-added depth information region based on the first type of key points to obtain at least one first region;
  • a second segmentation module used to triangulate the depth information template based on the second type of key points to obtain at least one second region
  • Laminating module correspondingly attaching the at least one second region to the at least one first region.
  • the information adding module includes:
  • a first distance calculation module calculating a distance between key points of a first type to obtain a first distance
  • a second distance calculation module used to calculate the distance between key points of the second type to obtain a second distance
  • Template adjustment module used to adjust the depth information template according to the ratio between the first distance and the second distance
  • Template information adding module used to add the adjusted depth information template to the pre-added depth information area.
  • an embodiment of the present disclosure further provides an electronic device.
  • the electronic device includes:
  • At least one processor At least one processor
  • a memory connected in communication with the at least one processor; wherein,
  • the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute the constructed image depth according to any one of the first aspects. Method of information.
  • an embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, where the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to cause a computer to execute any one of the claims of the first aspect.
  • the method for constructing image depth information is not limited to, but not limited to, but not limited to, but not limited to, but not limited to, but not limited to, but not limited to, but not limited to, but not limited to, a computer instructions, and the computer instructions are used to cause a computer to execute any one of the claims of the first aspect.
  • a method, an apparatus, an electronic device, and a non-transitory computer-readable storage medium for constructing image depth information provided by the embodiments of the present disclosure, wherein the image processing method includes: acquiring depth information related to a target object; marking the target object to pre-add depth information Area; add the acquired depth information to the pre-added depth information area.
  • the image processing method includes: acquiring depth information related to a target object; marking the target object to pre-add depth information Area; add the acquired depth information to the pre-added depth information area.
  • FIG. 1 is a flowchart of a method for constructing image depth information according to an embodiment of the present disclosure
  • FIG. 2 is a flowchart of marking a target object pre-added depth information region according to an embodiment of the present disclosure
  • FIG. 3 is a schematic diagram of converting values of an RGB color space into a two-dimensional vector containing only chroma and brightness according to an embodiment of the present disclosure
  • FIG. 4 is a flowchart of marking a region with pre-added depth information in foreground information according to an embodiment of the present disclosure
  • FIG. 5 is a schematic diagram of a corner extraction method provided by an embodiment of the present disclosure.
  • FIG. 6 is a flowchart of adding acquired depth information to an area where pre-added depth information is provided according to an embodiment of the present disclosure
  • FIG. 7 is a flowchart of adding depth information in a depth information template to a pre-added depth information area based on a first type of key point and a second type of key point according to an embodiment of the present disclosure
  • FIG. 8 is a flowchart of adding depth information in a depth information template to a pre-added depth information area based on a first type of key point and a second type of key point according to an embodiment of the present disclosure
  • FIG. 9 is a schematic block diagram of an apparatus for constructing image depth information according to an embodiment of the present disclosure.
  • FIG. 10 is a schematic block diagram of an area marking module according to an embodiment of the present disclosure.
  • FIG. 11 is a schematic block diagram of a foreground mark module according to an embodiment of the present disclosure.
  • FIG. 12 is a schematic block diagram of a foreground marking module according to an embodiment of the present disclosure.
  • FIG. 13 is a functional block diagram of an adding module according to an embodiment of the present disclosure.
  • 15 is a principle block diagram of an information adding module according to an embodiment of the present disclosure.
  • 16 is a schematic block diagram of an electronic device according to an embodiment of the present disclosure.
  • 17 is a schematic diagram of a computer-readable storage medium according to an embodiment of the present disclosure.
  • FIG. 18 is a schematic block diagram of a terminal according to an embodiment of the present disclosure.
  • the depth information includes a depth value.
  • the depth value specifically refers to the distance from an imaging point in the shooting scene to the XY plane where the camera center is located.
  • the XY plane refers to a plane parallel to the camera lens.
  • An embodiment of the present disclosure provides a method for constructing image depth information. As shown in FIG. 1, a method for constructing image depth information includes:
  • a target object for example, a human face
  • a target image without depth information does not appear to have a sense of hierarchy and three-dimensional.
  • depth information can be added to the target object.
  • the new depth information should match the target object.
  • depth information related to the target object In order to add depth information, it is necessary to obtain depth information related to the target object in advance.
  • image depth information There are many methods for obtaining image depth information, such as a 3D video shooting method based on a binocular camera or a camera array, a 3D video acquisition method based on a single camera mobile shooting, and so on. It can also be obtained through a camera with a depth information acquisition function.
  • the obtaining the depth information related to the target object may be directly downloading the depth information of the corresponding image on the network or locally, which may be determined according to the user's selection or the scene specifically selected by the user, such as the user can select the depth information of the face .
  • the user chooses to add depth information to the face, but in actual applications, the image collected by the camera is not only the image information of the face, but also background information, and picture information of the head, neck, or body associated with the face.
  • adding depth information it is mainly to add depth information to the face, and other parts (such as the head or neck) occupy a relatively small area, you do not need to add depth information, or if the background is relatively simple, you do not need to add depth information to it You only need to add depth information to the face, then you need to mark the face area, and not all areas of the face need to add depth information, so you can only apply to the eyes, cheeks, or nose according to the user's choice. Add depth information to other regions.
  • the depth information obtained in step S101 can be added to the marked area.
  • the addition at this point mainly refers to the use of digital signal processing to write the corresponding depth information to the corresponding area. region. That is, the depth information of the face obtained above is written into the pixels of the corresponding face image.
  • the area where the target object is pre-added depth information includes:
  • the foreground and background are separated, such as a photo containing a human face and a blue background, and the human face is separated from the blue background.
  • the foreground after the above separation is extracted, such as the foreground after the extraction in step S201, that is, the face.
  • Foreground and background separation that is, using mathematical methods to separate the foreground and background, and then extracting the separated foreground image, the background modeling method based on color information can be used to separate and extract the foreground.
  • the RGB color space value is converted into a two-dimensional vector containing only chrominance and brightness.
  • E i [E R (i), E G (i), E B (i)] represents the background construction.
  • the desired color of the background pixel i after the model, and the line segments OE i passing through the origins O and E i are called each other as desired chromaticity lines.
  • I i represents the RGB color values in the current image that you want to segment from the background. In order to better distinguish I i and E i , the following conversions have been made:
  • I i is the shortest distance OE i, i.e., a vertical line OE i I i, and this line segment OE i at point ⁇ i E i, ⁇ i represents the comparison background luminance desired value, the current brightness value of the pixel.
  • ⁇ i 1 means that the brightness value of the current pixel is expected to be the same as the background brightness value.
  • ⁇ i ⁇ 1 means that the current brightness value is lower than the background brightness value is expected.
  • ⁇ i > 1 indicates that the current brightness value is higher than the expected background brightness value.
  • ⁇ i is referred to as brightness deviation later.
  • I i is the shortest distance OE i, i.e., the point I i to segment CD i ⁇ i E i, the deviation with respect to a desired background color for the current pixel, is called color shift.
  • E i [u R (i), u G (i), u B (i)] is the color expectation of pixel i in N frames of background image
  • s i [ ⁇ R (i), ⁇ G (i) , ⁇ B (i)] is the color standard deviation of the pixel i in the N frame background image.
  • I R , I G , and I B are the components of the three RGB channels in I i .
  • Color values in the RGB color space are decomposed into brightness and chrominance.
  • By applying appropriate thresholds on the brightness deviation and chromaticity deviation of the pixel i to be judged it can be determined whether the current pixel is the foreground or the background. Because the RGB color space is converted into a two-dimensional space with only luminance deviation and chrominance deviation, it is possible to better judge whether the current pixel point is a shadow through the brightness deviation.
  • the specific division of pixel i is as follows:
  • the pixel is an ordinary background and is labeled B ′.
  • the pixel is the shadow of the foreground, labeled S.
  • the pixel is a background with high brightness (this may be caused by a change in lighting), and is labeled H.
  • this pixel is the foreground that needs to be extracted and is labeled F.
  • the different pixels in the video image are independent of each other, so the ⁇ i and CD i of the different pixels are subject to different distributions.
  • the ⁇ i and CD i of the pixels need to be Normalized.
  • Equation 3 The inverse is the normalized brightness deviation.
  • b i the standard deviation in the normalization.
  • the pixels can be classified into ordinary background (B '), shadow (S), bright background (H), and foreground (F).
  • the normalized classification formula is as follows:
  • ⁇ CD , ⁇ ⁇ 1 and ⁇ ⁇ 2 in Equation 5 are selected thresholds that determine the foreground and background.
  • marking a region with pre-added depth information in the foreground information includes:
  • the key points are the key points that can be obtained by using image processing technology and can represent the image features.
  • Area division can be divided into one area for each key point, or it can be divided into one area after combining multiple key points. For example, if the center of the eye can collect the key point, then the key point can be an area. Multiple points on the center and edge of the eye can be collected as key points, and then multiple key points on the center and edge of the eye are divided into one area. By analogy, the face can be divided into multiple regions.
  • marking the region of the pre-added depth information in the foreground information includes: extracting contour information of the foreground information. That is, after the foreground is extracted, such as a human face, an image processing algorithm is used to extract the contour information of the human face, thereby deleting parts such as the nose or eyes. This eliminates interference and reduces the amount of calculation.
  • Contour extraction uses the contour feature point extraction method, specifically the corner point extraction method in the contour feature point extraction method.
  • the method of corner extraction is as follows: a disc with a contour point as the center and a radius R is moved on the contour line, as shown in FIG. 5: when the disc is in the A position, the target and the background are in the disc The area of the target is half of the area of the disk. When the disk is in the B and C positions, the area of the target in the disk is less than half, and the area of the background in the disk is more than half. The area in the disc is more than half and the background in the disc is less than half.
  • the positions of B, C, and D in FIG. 5 are the positions of the corner points.
  • contour filling is performed first. For multi-connected targets, several closed contour lines will be obtained, and the contour lines will be filled one by one.
  • the target, background, and contour lines can be expressed by different values.
  • S ′ 0 (r 0 ) is the target in the disc with r 0 as the center
  • S ′ b (r 0 ) is r 0 Is the background in the center disk
  • S ′ c (r 0 ) is the area occupied by the contour line in the center of r 0.
  • the area occupied by the target and background including the contour is:
  • r 0 is marked as a candidate corner point only when ⁇ (r 0 ) ⁇ T 1 . If the contour point is approximated as the intersection of two straight line segments, ⁇ is the angle between the two straight line segments.
  • the radius R of the disc constitutes the support area for the measurement angle. Its selection mainly considers digitization, noise, and accuracy of the measurement angle. At present, most corner extraction methods use a group of adjacent contour points to calculate the curvature or the angle between two approximate straight line segments to determine the corner points. Since the contour points are involved in the calculation, they are susceptible to noise. In order to reduce the influence of noise, the usual approach is to obtain a larger support area and perform curve (straight line) fitting with a large amount of calculations at the same time. This leads to the algorithm not only being unable to adapt to small size, but also having a large amount of calculation. The algorithm of this embodiment uses the area enclosed by the contour line in the disk, and the area calculation is an integral operation.
  • the true corners are selected from a series of candidate corners obtained by Equation 7, and the true corners can be filtered through non-minimum suppression, that is, the candidate corners become corners only when the following formula holds:
  • r refers to any point in the disc except the center of the circle.
  • marking the region with pre-added depth information in the foreground includes: extracting contour information of the foreground; and marking the region with pre-added depth information within the contour information. That is, if the face information is extracted in the background, the facial features can be deleted, so that only the outline information of the face is retained, thereby reducing the amount of calculation.
  • the contour information of the foreground may be smoothed, that is, noise is eliminated, thereby making the contour smoother.
  • the depth information in the step of obtaining depth information related to the target object, is obtained from a depth information template.
  • the depth information in the template is set in advance. It can be a template generated by a technician after collecting with a camera with a depth information collection function, or it can be a depth information template obtained by calculating depth information based on a picture.
  • the local deep information template library can be updated after connecting to the network.
  • adding the acquired depth information to a pre-added depth information area includes:
  • S601 extract the key points of the area to which the depth information is pre-added, which are the first type of key points;
  • S602 Extract the key points of the depth information template, which are the second type of key points;
  • S603 Add the depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points.
  • the first type of key point may be one key point or multiple key points, which are mainly determined according to the depth information template and the degree of fit of the target object. For example, the area where the depth information is pre-added is a standard graphic, and one key point can be collected. If the area of the pre-added depth information is not standard, you need to collect multiple key points.
  • the first type of key points and the second type of key points both use the same method, which is a one-to-one correspondence.
  • adding depth information in a depth information template to a pre-added depth information area based on the first type of key points and the second type of key points includes:
  • S701 Triangulate the pre-added depth information area based on the first type of key points; obtain at least one first area;
  • S702 Triangulate the depth information template based on the second type of key points to obtain at least one second region.
  • the at least one second region is correspondingly attached to the at least one first region.
  • the triangulation is used. If the depth information template is divided into three parts: A, B, and C, the area where the depth information is pre-added is divided into A. The three parts ', B', and C 'are bonded to A', A to B ', and C to C'.
  • the key points of the edges and center points are more representative, when extracting the key points of the area with pre-added depth information, that is, when extracting the first type of key points, extract the key points at the edges and center points of the pre-added depth information point.
  • the key points at the edge and the center point of the pre-added depth information are specifically extracted.
  • the first type of key is There are multiple points and the second type of key points, and the number of the first type and the second type are the same.
  • the first type of key points and the second type of key points have a one-to-one correspondence. For example, if the area where the depth information is pre-added is the eye and the center of the eye is taken as the key point, then the center of the eye as the depth information template must also be the key point.
  • adding depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points includes:
  • S801 Calculate the distance between key points of the first type to obtain the first distance
  • the depth information template is preset, but the target object is changed. Different target objects have different parameters. Therefore, it is necessary to adjust the applicability of the depth information template, such as the depth information of the face containing the depth information of the face.
  • the template when adding depth information, adding different objects to different people, then the size of their faces will be different. If there are larger faces and some smaller faces, larger faces or smaller faces It cannot be adapted to the template, so the depth information needs to be adjusted according to the size of the face, that is, the template is correspondingly reduced or enlarged according to the size of the face.
  • the second type of key point is correspondingly a * b * c *. Then the length of the line between a and b, the length of the line between b and c, and the length of the line between c and a is the first distance, then the line between a * and b * , The length of the line between b * and c *, and the length of the line between c * and a * is the second distance.
  • the ratio of the first distance to the second distance is a scale. The ratio may also be based on the length of the line between a and b as the first distance, the length of the line between a * and b * as the second distance, and so on.
  • the apparatus includes:
  • the obtaining module 901 is configured to obtain depth information related to the target object; the obtaining module is used to obtain depth information related to the target object, specifically:
  • An area labeling module 902 configured to mark an area in which target objects are pre-added depth information
  • the adding module 903 is configured to add the acquired depth information to a region in which the depth information is pre-added.
  • the area marking module includes:
  • Separation module 1001 for separating the foreground and background of the target object
  • Extraction module 1002 for extracting the foreground of the target object
  • Foreground labeling module 1003 used for labeling a region with pre-added depth information in the foreground.
  • the foreground mark module includes:
  • Key point extraction module 1101 used to extract key points in the foreground
  • An area division module 1102 configured to divide the foreground based on the key points
  • Keypoint marking module 1103 It is used to mark the keypoints of the pre-added depth information.
  • the foreground mark module includes:
  • a contour extraction module 1201 configured to extract contour information of the foreground
  • Contour marking module 1202 used to mark a region with pre-added depth information within the contour information.
  • it further includes:
  • the smoothing processing module 1203 is configured to smooth the contour information extracted by the contour extraction module.
  • adding a module includes:
  • the first type of key point extraction module 1301 the key points of the area for extracting pre-added depth information are the first type of key points;
  • Keypoint extraction module 1302 of the second type keypoints for extracting depth information templates, which are keypoints of the second type;
  • the information adding module 1303 is configured to add the depth information in the depth information template to the pre-added depth information area based on the first type of key points and the second type of key points.
  • the information adding module includes:
  • the first segmentation module 1401 is configured to perform triangulation on the pre-added depth information region based on the first type of key points to obtain at least one first region;
  • a second segmentation module 1402 configured to triangulate the depth information template based on the second type of key points to obtain at least one second region;
  • the bonding module 1403 is configured to correspondingly bond the at least one second region to the at least one first region.
  • the information adding module includes:
  • First distance calculation module 1501 calculating a distance between key points of a first type to obtain a first distance
  • a second distance calculation module 1502 calculating a distance between key points of a second type to obtain a second distance
  • Template adjustment module 1503 configured to adjust the depth information template according to the ratio between the first distance and the second distance;
  • Template information adding module 1504 It is used to add the adjusted depth information template to the pre-added depth information area.
  • FIG. 16 is a hardware block diagram illustrating an electronic device according to an embodiment of the present disclosure. As shown in FIG. 16, the electronic device 160 according to an embodiment of the present disclosure includes a memory 161 and a processor 162.
  • the memory 161 is configured to store non-transitory computer-readable instructions.
  • the memory 161 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory.
  • the volatile memory may include, for example, a random access memory (RAM) and / or a cache memory.
  • the non-volatile memory may include, for example, a read-only memory (ROM), a hard disk, a flash memory, and the like.
  • the processor 162 may be a central processing unit (CPU) or other form of processing unit having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 160 to perform desired functions.
  • the processor 162 is configured to run the computer-readable instructions stored in the memory 161, so that the electronic device 160 executes all or part of the image depth information constructed by the foregoing embodiments of the present disclosure. step.
  • this embodiment may also include well-known structures such as a communication bus and an interface. These well-known structures should also be included in the protection scope of the present disclosure within.
  • FIG. 17 is a schematic diagram illustrating a computer-readable storage medium according to an embodiment of the present disclosure.
  • a computer-readable storage medium 170 according to an embodiment of the present disclosure stores non-transitory computer-readable instructions 171 thereon.
  • the non-transitory computer-readable instructions 171 are executed by a processor, all or part of the steps of constructing image depth information of the foregoing embodiments of the present disclosure are performed.
  • the computer-readable storage medium 170 includes, but is not limited to, optical storage media (for example, CD-ROM and DVD), magneto-optical storage media (for example, MO), magnetic storage media (for example, tape or mobile hard disk), Non-volatile memory rewritable media (for example: memory card) and media with built-in ROM (for example: ROM box).
  • optical storage media for example, CD-ROM and DVD
  • magneto-optical storage media for example, MO
  • magnetic storage media for example, tape or mobile hard disk
  • Non-volatile memory rewritable media for example: memory card
  • media with built-in ROM for example: ROM box
  • FIG. 18 is a schematic diagram illustrating a hardware structure of a terminal device according to an embodiment of the present disclosure. As shown in FIG. 18, the terminal 180 includes the foregoing embodiment of the apparatus for constructing image depth information.
  • the terminal device may be implemented in various forms, and the terminal device in the present disclosure may include, but is not limited to, such as a mobile phone, a smart phone, a notebook computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), Mobile terminal devices such as PMPs (portable multimedia players), navigation devices, vehicle-mounted terminal devices, vehicle-mounted display terminals, vehicle-mounted electronic rear-view mirrors, and the like, and fixed terminal devices such as digital TVs, desktop computers, and the like.
  • PMPs portable multimedia players
  • navigation devices such as PMPs (portable multimedia players), navigation devices, vehicle-mounted terminal devices, vehicle-mounted display terminals, vehicle-mounted electronic rear-view mirrors, and the like
  • fixed terminal devices such as digital TVs, desktop computers, and the like.
  • the terminal 180 may further include other components. As shown in FIG. 7, the terminal 180 may include a power supply unit 181, a wireless communication unit 182, an A / V (audio / video) input unit 183, a user input unit 184, a sensing unit 185, an interface unit 186, a controller 187, The output unit 188 and the storage unit 189 and so on.
  • FIG. 18 shows a terminal having various components, but it should be understood that it is not required to implement all of the illustrated components, and more or fewer components may be implemented instead.
  • the wireless communication unit 182 allows radio communication between the terminal 180 and a wireless communication system or network.
  • the A / V input unit 183 is used to receive audio or video signals.
  • the user input unit 184 may generate key input data according to a command input by the user to control various operations of the terminal device.
  • the sensing unit 185 detects the current status of the terminal 180, the position of the terminal 180, the presence or absence of a user's touch input to the terminal 180, the orientation of the terminal 180, the acceleration or deceleration movement and direction of the terminal 180, and the like, and generates a terminal for controlling the terminal Command or signal of 180 operations.
  • the interface unit 186 serves as an interface through which at least one external device can connect with the terminal 180.
  • the output unit 188 is configured to provide an output signal in a visual, audio, and / or tactile manner.
  • the storage unit 189 may store software programs and the like for processing and control operations performed by the controller 187, or may temporarily store data that has been output or is to be output.
  • the storage unit 189 may include at least one type of storage medium.
  • the terminal 180 may cooperate with a network storage device that performs a storage function of the storage unit 189 through a network connection.
  • the controller 187 generally controls the overall operation of the terminal device.
  • the controller 187 may include a multimedia module for reproducing or playing back multimedia data.
  • the controller 187 may perform a pattern recognition process to recognize a handwriting input or a picture drawing input performed on the touch screen as characters or images.
  • the power supply unit 181 receives external power or internal power under the control of the controller 187 and provides appropriate power required to operate various elements and components.
  • Various embodiments of constructing image depth information proposed by the present disclosure may be implemented using a computer-readable medium, such as computer software, hardware, or any combination thereof.
  • various embodiments of the video feature comparison method proposed in the present disclosure can be implemented by using an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), field programmable gate array (FPGA), processor, controller, microcontroller, microprocessor, electronic unit designed to perform the functions described herein, and in some cases
  • ASIC application specific integrated circuit
  • DSP digital signal processor
  • DSPD digital signal processing device
  • PLD programmable logic device
  • FPGA field programmable gate array
  • processor controller, microcontroller, microprocessor, electronic unit designed to perform the functions described herein, and in some cases
  • controller 187 Various embodiments of the video feature comparison method proposed in the present disclosure may be implemented in the controller 187.
  • various embodiments of the video feature comparison method proposed by the present disclosure may be implemented with a separate software module that allows performing at least one function or operation.
  • the software codes may be implemented by a software application (or program) written in any suitable programming language, and the software codes may be stored in the storage unit 189 and executed by the controller 187.
  • relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that any such relationship exists between these entities or operations.
  • the block diagrams of the devices, devices, devices, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As those skilled in the art would realize, these devices, devices, equipment, and systems may be connected, arranged, and configured in any manner. Words such as “including,” “including,” “having,” and the like are open words, meaning “including, but not limited to,” and can be used interchangeably with them.
  • an "or” used in an enumeration of items beginning with “at least one” indicates a separate enumeration such that, for example, an "at least one of A, B, or C” enumeration means A or B or C, or AB or AC or BC, or ABC (ie A and B and C).
  • the word "exemplary” does not mean that the described example is preferred or better than other examples.
  • each component or each step can be disassembled and / or recombined.
  • These decompositions and / or recombinations should be considered as equivalent solutions of the present disclosure.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本公开实施例公开了一种构建图像深度信息的方法及装置，涉及图像处理领域。其中，构建图像深度信息的方法，包括：获取与目标对象相关的深度信息；标记目标对象预添加深度信息的区域；将获取的深度信息添加到预添加深度信息的区域。通过将预设的深度信息添加到图像中，从而使现有不具有深度信息的图像采集装置获取的图像，富有立体感。

Description

构建图像深度信息的方法及装置
本申请要求于2018年6月13日提交中国专利局、申请号为201810619716.8，发明名称为“构建图像深度信息的方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本公开涉及图像处理领域，尤其涉及一种构建图像深度信息的方法及装置。
背景技术
随着网络和硬件的发展，人们通过智能终端来记录生活已被广泛应用，现如今，可用于拍照的移动终端也越来越多，如手机、平板电脑等移动终端，且随着智能终端摄像头像素的不断提高，现在的手机已经取代传统的照相机等，成为生活常用的拍摄设备。而在设备，如手机中安装各种图像处理软件或插件，对拍摄的图像或视频进行美颜或添加贴纸等也广泛被用户使用。
发明内容
但现有的终端中或处于成本的考虑，大多配置的摄像头不具有采集深度信息的能力，尤其是前置摄像头，更加不具有采集深度信息的能力，使得终端采集的图像缺乏深度信息而缺乏立体感。
有鉴于此，本公开实施例提供了一种构建图像深度信息的方法及装置，至少部分的解决现有技术中存在的问题。
第一方面，本公开实施例提供了一种构建图像深度信息的方法，包括：
获取与目标对象相关的深度信息；
标记目标对象预添加深度信息的区域；
将获取的深度信息添加到预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，所述标记目标对象预添加深度信息的区域，包括：
分离所述目标对象的前景和背景；
提取所述目标对象的前景；
在所述前景中标记预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，所述在所述前景信息中标记预添加深度信息的区域，包括：
提取所述前景中的关键点；
基于所述关键点对所述前景进行区域划分；
标记预添加深度信息的区域的关键点。
作为本公开实施例的一种具体实现方式，所述在所述前景中标记预添加深度信息的区域，包括：
提取所述前景的轮廓信息；
在所述轮廓信息内标记预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，在提取所述前景的轮廓信息后，还包括：
对所述轮廓信息进行平滑处理。
作为本公开实施例的一种具体实现方式，所述获取与目标对象相关的深度信息，具体为：
从深度信息模板中获取与目标对象相关的深度信息。
作为本公开实施例的一种具体实现方式，所述将获取的深度信息添加到预添加深度信息的区域，包括：
提取预添加深度信息的区域的关键点，为第一类关键点；
提取深度信息模板的关键点，为第二类关键点；
基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，所述基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
基于第一类关键点对预添加深度信息的区域进行三角剖分，得到至少一个第一区域；
基于第二类关键点对深度信息模板进行三角剖分，得到至少一个第二区域；
将所述至少一个第二区域对应贴合到所述至少一个第一区域。
作为本公开实施例的一种具体实现方式，所述基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
计算第一类关键点之间的距离，得出第一距离；
计算第二类关键点之间的距离，得出第二距离；
根据第一距离和第二距离之间的比值，对深度信息模板进行调整；
将调整后的深度信息模板添加到预添加深度信息的区域。
第二方面，本公开实施例还提供了一种构建图像深度信息的装置，包括：
获取模块，用于获取与目标对象相关的深度信息；
区域标记模块，用于标记目标对象预添加深度信息的区域；
添加模块，用于将获取的深度信息添加到预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，
所述区域标记模块包括：
分离模块：用于分离所述目标对象的前景和背景；
提取模块：用于提取所述目标对象的前景；
前景标记模块：用于在所述前景中标记预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，所述前景标记模块，包括：
关键点提取模块：用于提取所述前景中的关键点；
区域划分模块：用于基于所述关键点对所述前景进行区域划分；
关键点标记模块：用于标记预添加深度信息的区域的关键点。
作为本公开实施例的一种具体实现方式，
所述前景标记模块，包括：
轮廓提取模块：用于提取所述前景的轮廓信息；
轮廓标记模块：用于在所述轮廓信息内标记预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，
还包括：
平滑处理模块：用于对所述轮廓提取模块提取的所述轮廓信息进行平滑处理。
作为本公开实施例的一种具体实现方式，所述获取模块用于获取与目标对象相关的深度信息，具体为：
从深度信息模板中获取与目标对象相关的深度信息。
作为本公开实施例的一种具体实现方式，
所述添加模块，包括：
第一类关键点提取模块：用于提取预添加深度信息的区域的关键点，为第一类关键点；
第二类关键点提取模块：用于提取深度信息模板的关键点，为第二类关键点；
信息添加模块：用于基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域。
作为本公开实施例的一种具体实现方式，所述信息添加模块，包括：
第一剖分模块：用于基于第一类关键点对预添加深度信息的区域进行三角剖分，得到至少一个第一区域；
第二剖分模块：用于基于第二类关键点对深度信息模板进行三角剖分，得到至少一个第二区域；
贴合模块：用于将所述至少一个第二区域对应贴合到所述至少一个第一区域。
作为本公开实施例的一种具体实现方式，所述信息添加模块，包括：
第一距离计算模块：用于计算第一类关键点之间的距离，得出第一距离；
第二距离计算模块：用于计算第二类关键点之间的距离，得出第二距离；
模板调整模块：用于根据第一距离和第二距离之间的比值，对深度信息模板进行调整；
模板信息添加模块：用于将调整后的深度信息模板添加到预添加深度信息的区域。
第三方面，本公开实施例还提供了一种电子设备，该电子设备包括：
至少一个处理器；以及，
与所述至少一个处理器通信连接的存储器；其中，
所述存储器存储有能被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行第一方面任一所述的构建图像深度信息的方法。
第四方面，本公开实施例还提供了一种非暂态计算机可读存储介质，该非暂态计算机可读存储介质存储计算机指令，该计算机指令用于使计算机执行权利要求第一方面任一所述的构建图像深度信息的方法。
本公开实施例提供的构建图像深度信息的方法、装置、电子设备及非暂态计算机可读存储介质，其中该图像处理方法包括：获取与目标对象相关的深度信息；标记目标对象预添加深度信息的区域；将获取的深度信息添加到预添加深度信息的区域。本公开实施例，通过将预设的深度信息添加到图像中，从而使现有不具有深度信息的图像采集装置获取的图像，富有立体感。
上述说明仅是本公开技术方案的概述，为了能更清楚了解本公开的技术手段，而可依照说明书的内容予以实施，并且为让本公开的上述和其他目的、特征和优点能够更明显易懂，以下特举较佳实施例，并配合附图，详细说明如下。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其它的附图。
图1为本公开实施例提供的构建图像深度信息的方法的流程图；
图2为本公开实施例提供的标记目标对象预添加深度信息的区域的流程图；
图3为本公开实施例提供的把RGB颜色空间的值转换为只含有色度和亮度的二维向量的示意图；
图4为本公开实施例提供的在前景信息中标记预添加深度信息的区域的流程图；
图5为本公开实施例提供的角点提取方法中的示意图；
图6为本公开实施例提供的将获取的深度信息添加到预添加深度信息的区域的流程图；
图7为本公开实施例提供的基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域的流程图；
图8为本公开实施例提供的基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域的流程图；
图9为本公开实施例提供的构建图像深度信息的装置的原理框图；
图10为本公开实施例提供的区域标记模块的原理框图；
图11为本公开实施例提供的前景标记模块的原理框图；
图12为本公开实施例提供的前景标记模块的原理框图；
图13为本公开实施例提供的添加模块的原理框图；
图14为本公开实施例提供的信息添加模块的原理框图；
图15为本公开实施例提供的信息添加模块的原理框图；
图16为本公开实施例提供的电子设备的原理框图；
图17为本公开实施例提供的计算机可读存储介质的示意图；
图18为本公开实施例提供的终端的原理框图。
具体实施方式
下面结合附图对本公开实施例进行详细描述。
应当明确，以下通过特定的具体实例说明本公开的实施方式，本领域技术人员可由本说明书所揭露的内容轻易地了解本公开的其他优点与功效。显然，所描述的实施例仅仅是本公开一部分实施例，而不是全部的实施例。本公开还可以通过另外不同的具体实施方式加以实施或应用，本说明书中的各项细节也可以基于不同观点与应用，在没有背离本公开的精神下进行各种修饰或改变。需说明的是，在不冲突的情况下，以下实施例及实施例中的特征可以相互组合。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
需要说明的是，下文描述在所附权利要求书的范围内的实施例的各种方面。应显而易见，本文中所描述的方面可体现于广泛多种形式中，且本文中所描述的任何特定结构及/或功能仅为说明性的。基于本公开，所属领域的技术人员应了解，本文中所描述的一个方面可与任何其它方面独立地实施，且可以各种方式组合这些方面中的两者或两者以上。举例来说，可使用本文中所阐述的任何数目个方面来实施设备及/或实践方法。另外，可使用除了本文中所阐述的方面中的一或多者之外的其它结构及/或功能性实施此设备及/或实践此方法。
还需要说明的是，以下实施例中所提供的图示仅以示意方式说明本公开的基本构想，图式中仅显示与本公开中有关的组件而非按照实际实施时的组件数目、形状及尺寸绘制，其实际实施时各组件的型态、数量及比例可为一种随意的改变，且其组件布局型态也可能更为复杂。
另外，在以下描述中，提供具体细节是为了便于透彻理解实例。然而，所属领域的技术人员将理解，可在没有这些特定细节的情况下实践所述方面。
深度信息里面包括深度值，深度值具体是指拍摄场景中某个成像点到相机中心所在的XY平面的距离，XY平面是指与相机的镜头平行的平面。
本公开实施例提供一种构建图像深度信息的方法。如图1所示，构建图像深度信息的方法，包括：
S101:获取与目标对象相关的深度信息；
常见的图像在经过图像处理之后，图像中的目标对象(例如，人脸)通常不含有深度信息，没有深度信息的目标图像看起来不具有层次感和立体感。
为了使的目标对象看起来更有层次感和立体感，可以为目标对象增加深度信息。同时，新增的深度信息应当是与目标对象匹配。
为了新增深度信息，需要预先获取与目标对象相关的深度信息。获得图像深度信息方法比较多，如基于双目摄像头或摄像头阵列来实现3D视频的拍摄方法，基于单摄像头移动拍摄获取3D视频的方法等等，也可以通过具有深度信息采集功能的摄像头获取。
该获取与目标对象相关的深度信息可以是在网络上直接下载或从本地获取相应图像的深度信息，其可以根据用户的选择或用户具体选择的场景来确定，如用户可以选择人脸的深度信息。
S102:标记目标对象预添加深度信息的区域；
如用户选择对人脸添加深度信息，但实际应用中摄像头采集的图像不只有人脸的图像信息，还具有背景信息，以及与人脸关联的头部、颈部或者身体的图片信息，但在添加深度信息时主要是为人脸添加深度信息，而其它的部分(如头部或颈部)占比较小，则不需要添加深度信息，或者如背景比较单一，也不需要对其添加深度信息，而只需要对人脸添加深度信息，那么就需要将人脸区域标记出来，而对于人脸也并不是所有的区域都需要添加深度信息，因此也可以根据用户的选择只对眼睛、脸颊或者鼻子等区域部分添加深度信息。
S103:将获取的深度信息添加到预添加深度信息的区域。
在标记过需要添加深度信息的区域后，将步骤S101获取的深度信息添加到标记的区域即可，该处的添加，主要指采用数字信号处理的方式，将相应的深度信息写入到相应的区域。即将上述获取的人脸的深度信息，写入相应的人脸图像的像素点。
根据本公开另一实施例，如图2所示，标记目标对象预添加深度信息的区域包括，
S201:分离所述目标对象的前景和背景；
即将前景和背景分离，如包含人脸和蓝色背景的照片，将人脸和蓝色背景分离。
S202:提取所述目标对象的前景；
即将上面分离后的前景提取出来，如提取步骤S201分离后的前景，即如人脸。
S203:在所述前景中标记预添加深度信息的区域。
如果只需要在人脸的面颊部分添加深度信息，则将面颊部分标记出来，如果需要对人脸的鼻子或眼眶添加深度信息，则相应的将鼻子或眼眶标记出来。
前景和背景分离，即使用数学方法将前景和背景分离开，然后将分离后的前景图像提取出来，可采用基于颜色信息的背景建模方法分离并提取前景，
基于颜色信息的背景建模方法具体如下：
把RGB颜色空间的值转换为只含有色度和亮度的二维向量，如图3所示， E i＝[E R(i)，E G(i)，E B(i)]代表背景建模后背景像素i颜色的期望，经过原点O和E i的线段OE i互被称为期望色度线。I i则表示想从背景分割的当前图像中的RGB颜色值，为了更好区分I i和E i，做了以下的转换：
I i到OE i的最短距离，即I i到OE i的垂线，此线与线段OE i相交于点α iE i，α i代表与背景亮度期望值比较，当前像素的亮度值。α i＝1表示当前像素的亮度值与背景亮度值期望相同。α i＜1表示当前亮度值比背景亮度值期望低。α i＞1表示当前亮度值比背景亮度期望值高。为表述方便，后面称α i为亮度偏差。
I i到OE i的最短距离，即点I i到α iE i的线段CD i，为当前像素相对于背景颜色期望的偏差，称之为颜色偏差。
定义E i＝[u R(i)，u G(i)，u B(i)]为N帧背景图像中像素i的颜色期望，s i＝[σ R(i)，σ G(i)，σ B(i)]为N帧背景图像中像素i的颜色标准差。由此可以得到亮度偏差α i，和颜色偏差CD i，的计算公式如下：
Figure PCTCN2019073070-appb-000001
Figure PCTCN2019073070-appb-000002
I R，I G，I B为I i中RGB三个通道的分量。
RGB颜色空间的颜色值被分解成亮度和色度。在需要判断的像素i的亮度偏差和色度偏差上应用恰当的阀值，就能够确定当前像素是前景还是背景。因为RGB颜色空间被转化为只有亮度偏差和色度偏差的二维空间，所以能够较好的通过亮度偏差判断当前像素点是否为阴影。对像素i的具体划分如下：
如果当前像素与背景像素期望具有相似的亮度和色度(亮度偏差和色度偏差在一定范围内)，则像素为普通背景，标记为B′。
如果当前像素与背景像素期望具有相似的色度，但是亮度比背景像素的亮度值低，则像素为前景的阴影，标记为S。
如果当前像素与背景像素期望具有相似的色度，但是亮度比背景像素的亮度值高，则像素为具有高亮度的背景(此可能由于光照发生变化造成)，标记为H。
如果当前像素与背景像素期望具有不同的色度，则此像素为需要提取的前景，标记为F。
视频图像中不同的像素点是相互独立的，因此不同像素点的α i和CD i，服从不同的分布，为了对不同的像素点运用统一的阀值，需要对像素点的α i和CD i归一化。
Figure PCTCN2019073070-appb-000003
Figure PCTCN2019073070-appb-000004
公式3中
Figure PCTCN2019073070-appb-000005
的反为归一化的亮度偏差，公式4中的
Figure PCTCN2019073070-appb-000006
为归一化的色度偏差，b i是归一化中的标准差。
因此，像素点可以被归为普通背景(B′)、阴影(S)、亮背景(H)和前景(F)。
经过归一化之后的分类公式如下：
Figure PCTCN2019073070-appb-000007
公式5中的τ CD，τ α1和τ α2是选中的决定前景背景的阀值。
根据公式1、2、3、4和5对N帧背景图像建立模型，具体步骤如下：
(a)求出N帧背景图像的期望，标准差；
(b)由a可继续求得N帧图像的颜色偏差，亮度偏差；
(c)对颜色偏差，亮度偏差标准化；
(d)对N帧图像的标准颜色偏差和标准亮度偏差进行统计，设置判定前景背景的阀值，背景建模完成；
(e)对需要进行前景提取的图像同样求得标准颜色偏差和标准亮度偏差，
然后根据由d得出的阀值判断是前景还是背景。
根据本公开另一实施例，如图4所示，在前景信息中标记预添加深度信息的区域，包括：
S401:提取前景中的关键点；
提取前景中的关键点，如人脸的关键点，关键点即采用图像处理技术获得的能够代表图像特征的关键点。
S402:基于关键点对前景进行区域划分；
区域划分可以是每个关键点划分为一个区域，也可以是将多个关键点组合后划分为一个区域，如眼睛可以采集眼睛的中心为关键点，那么该关键点即可为一个区域，也可以采集眼睛中心和边缘上的多个点为关键点，那么将眼睛上的中心和边缘上的多个关键点划分为一个区域。依次类推，从而可以将脸部划分为多个区域。
S403:标记预添加深度信息的区域的关键点。
将脸划分为多个区域后，对需要添加深度信息的区域内的关键点进行标记，如眼睛需要添加深度信息，则将眼睛区域的关键点标记为1，其它不需要添加深度信息的关键点标记为0。
根据本公开另一实施例，在前景信息中标记预添加深度信息的区域，包括：提取前景信息的轮廓信息。即在提取出前景后，如人脸后，采用图像处理算法，将人脸的轮廓信息提取出来，从而将鼻子或眼睛等部位删除。从而排除干扰因素，也能减少计算量。
轮廓提取，采用轮廓特征点的提取方法，具体采用轮廓特征点的提取方法中的角点提取方法。
角点提取方法，具体为：设有一个以轮廓点为中心、半径为R的圆盘在轮廓线上移动，如图5所示：当圆盘位于A位置时，目标和背景在圆盘中的面积是圆盘面积的一半；当圆盘位于B、C位置时，目标在圆盘中的面积小于一半，背景在圆盘中的面积大于一半；当圆盘位于D位置时，目标在圆盘中的面积大于一半而背景在圆盘中的面积小于一半。图5中B、C、D的位置是角点的位置。基于上述观察，可有如下结论：当圆盘处在直线上时，目标和背景在圆盘中的面积是圆盘面积的一半；当角点处在圆盘中时，目标和背景在圆盘中的面积总有一个小于圆盘面积的一半。根据这一点可以进行角点检测。
为区别目标区和背景区(无须确定出哪个是真正的目标区，哪个是真正的背景区， 只需区别出这两个区域即可)，首先进行轮廓填充。对于多连通目标，会得到若干个闭合轮廓线，将逐个填充轮廓线。在算法具体实现时，可将目标、背景及轮廓线用不同的数值表示，S′ 0(r 0)为以r 0为中心圆盘内的目标，S′ b(r 0)为以r 0为中心圆盘内的背景，S′ c(r 0)为以r 0为中心圆盘内的轮廓线所占的面积，则包含轮廓的目标和背景所占的面积为：
Figure PCTCN2019073070-appb-000008
它们满足S o(r 0)+S b(r 0)＝S d，其中S d是圆盘的面积，即圆盘内点的总数。令S min(r 0)＝min{S o(r 0)，S′ b(r 0)}，     公式(7)，
则当
Figure PCTCN2019073070-appb-000009
时，将r 0标志为候选角点，公式7中的T 1是阈值。令
Figure PCTCN2019073070-appb-000010
则由公式7可知：只有当θ(r 0)≤T 1时，才将r 0标志为候选角点。若将轮廓点近似看作是两直线段的交点，则θ即为这两直线段的夹角。
圆盘半径R构成了测量角度的支撑区域，它的选取主要考虑数字化、噪声及测量角度的精度。目前绝大多数角点提取方法都是利用一组相邻的轮廓点来计算曲率或两近似直线段的夹角来判定角点的，由于参与计算的是轮廓点，所以易受噪声的影响。为了减小噪声的影响，通常的途径是将支撑区域取得较大，同时进行运算量较大的曲线(直线)拟合。这样就导致了算法不仅对小尺寸不能适应而且运算量也较大。本实施例的算法是利用轮廓线在圆盘中所围成的面积，求面积是积分运算。
然后从由公式7得到的一系列候选角点中筛选出真正的角点，通过非极小抑制即可筛选出真正的角点，即只有当下式成立时，候选角点才成为角点：
Figure PCTCN2019073070-appb-000011
r指代的就是圆盘内的除圆心外的任意一个点。
然后将筛选出的真正的角点连线，即得出轮廓信息。
根据本公开另一实施例，前景中标记预添加深度信息的区域，包括：提取前景的轮廓信息；在轮廓信息内标记预添加深度信息的区域。即如将人脸信息在背景中提取后，可将五官删除，从而只保留人脸的轮廓信息，从而减少计算量。
根据本公开另一实施例，在提取前景的轮廓信息的后，可以对轮廓信息进行平滑处理，即排除噪音，从而使轮廓更加的平滑。
根据本公开另一实施例，获取与目标对象相关的深度信息的步骤中，深度信息是从深度信息模板中获取的。模板中的深度信息是预先设定好的，其可以是技术人员利用具有深度信息采集功能的摄像头采集后，生成的模板，也可以是根据图片进行深度信息计算获取的深度信息模板。该深度信息模板，在使用时，可从网络上下载相应的模板，也可以使用已经保存到本地的深度信息模板，如对人脸添加深度信息时，可以在网络上查找人脸的深度信息模板，也可以在本地存储的深度信息模板中查找人脸深度信息模板。且本地的深度信息模板库可以在连接到网络后进行更新。
根据本公开另一实施例，如图6所示，将获取的深度信息添加到预添加深度信息的区域，包括：
S601:提取预添加深度信息的区域的关键点，为第一类关键点；
S602:提取深度信息模板的关键点，为第二类关键点；
S603:基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域。
第一类关键点可能为一个关键点也可能为多个关键点，主要根据深度信息模板以及目标对象的契合度来确定，如预添加深度信息的区域就是标准图形，采集一个关键点即可，如预添加深度信息的区域形状不标准，则需要采集多个关键点。第一类关键点和第二类关键点都是采用相同的方式，其为一一对应的关系。
根据本公开另一实施例，如图7所示，基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
S701:基于第一类关键点对预添加深度信息的区域进行三角剖分；得到至少一个第一区域；
S702:基于第二类关键点对深度信息模板进行三角剖分，得到至少一个第二区域；
S703:将所述至少一个第二区域对应贴合到所述至少一个第一区域。
如当目标对象比较复杂则不规则时，则采用三角剖分的形式，如将深度信息模板剖分为A、B、C三个部分，则预添加深度信息的区域则相应的剖分为A’、B’、C’三个部分，贴合时将A贴合到A’，将B贴合到B’，将C贴合到C’。
因边缘和中心点的关键点比较有代表性，因此提取预添加深度信息的区域的关键点时，即提取第一类关键点时，具体提取预添加深度信息的区域边缘和中心点处的关键点。
相应的提取预添加深度信息的区域的关键点时，即提取第二类关键点时，具体提取预添加深度信息的区域边缘和中心点处的关键点。
并且在提取关键点时，在预添加深度信息的区域形状不标准时，如只提取一个关键点，其不能很好的将深度信息模板和预添加深度信息的区域契合在一起，因此第一 类关键点和第二类关键点均为多个，且第一类关键点和第二类关键点个数相同。
第一类关键点和第二类关键点为一一对应，如预添加深度信息的区域为眼睛，取眼睛的中心为关键点，那么对作为深度信息模板的眼睛的中心也要为关键点。
根据本公开另一实施例，如图8所示，基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
S801:计算第一类关键点之间的距离，得出第一距离；
S802:计算第二类关键点之间的距离，得出第二距离；
S803:根据第一距离和第二距离之间的比值，对深度信息模板进行调整。
因深度信息模板为预设好的，但目标对象是变化的，不同的目标对象，其参数存在差异，因此需要对深度信息模板进行适用性调整，如对于包含人脸深度信息的人脸深度信息模板，在添加深度信息时，添加对象为不同的人，那么其人脸的大小也就不同，如有的人脸较大而有的人脸较小，人脸较大或人脸较小都不能和模板相适配，因此需要根据人脸的大小对深度信息进行调整，即根据人脸的大小对模板进行相应的缩小或放大。
如第一类关键点为abc，则第二类关键点相应的为a*b*c*。则a和b之间的连线的长度，b和c之间的连线的长度，c和a之间的连线的长度即为第一距离，则a*和b*之间的连线的长度，b*和c*之间的连线的长度，c*和a*之间的连线的长度即为第二距离，将第一距离和第二距离进行比值，即得出一个缩放比例，也可以只取a和b之间的连线的长度为第一距离，取a*和b*的之间连线的长度为第二距离等等。
相应的本公开技术方案还公开了一种构建图像深度信息的装置，如图9所示，装置，包括：
获取模块901，用于获取与目标对象相关的深度信息；获取模块用于获取与目标对象相关的深度信息，具体为：
从深度信息模板中获取与目标对象相关的深度信息。
区域标记模块902，用于标记目标对象预添加深度信息的区域；
添加模块903，用于将获取的深度信息添加到预添加深度信息的区域。
优选的，如图10所示，区域标记模块包括：
分离模块1001：用于分离所述目标对象的前景和背景；
提取模块1002：用于提取所述目标对象的前景；
前景标记模块1003：用于在所述前景中标记预添加深度信息的区域。
优选的，如图11所示，前景标记模块，包括：
关键点提取模块1101：用于提取所述前景中的关键点；
区域划分模块1102：用于基于所述关键点对所述前景进行区域划分；
关键点标记模块1103：用于标记预添加深度信息的区域的关键点。
优选的，如图12所示，前景标记模块，包括：
轮廓提取模块1201：用于提取所述前景的轮廓信息；
轮廓标记模块1202：用于在所述轮廓信息内标记预添加深度信息的区域。
优选的，还包括：
平滑处理模块1203：用于对轮廓提取模块提取的所述轮廓信息进行平滑处理。
优选的，如图13所示，添加模块，包括：
第一类关键点提取模块1301：用于提取预添加深度信息的区域的关键点，为第一类关键点；
第二类关键点提取模块1302：用于提取深度信息模板的关键点，为第二类关键点；
信息添加模块1303：用于基于第一类关键点和第二类关键点，将深度信息模板中的深度信息添加到预添加深度信息的区域。
优选的，如图14所示，信息添加模块，包括：
第一剖分模块1401：用于基于第一类关键点对预添加深度信息的区域进行三角剖分，得到至少一个第一区域；
第二剖分模块1402：用于基于第二类关键点对深度信息模板进行三角剖分，得到至少一个第二区域；
贴合模块1403：用于将所述至少一个第二区域对应贴合到所述至少一个第一区域。
优选的，如图15所示，信息添加模块，包括：
第一距离计算模块1501：用于计算第一类关键点之间的距离，得出第一距离；
第二距离计算模块1502：用于计算第二类关键点之间的距离，得出第二距离；
模板调整模块1503：用于根据第一距离和第二距离之间的比值，对深度信息模板进行调整；
模板信息添加模块1504：用于将调整后的深度信息模板添加到预添加深度信息的区域。
图16是图示根据本公开的实施例的电子设备的硬件框图。如图16所示，根据本公开实施例的电子设备160包括存储器161和处理器162。
该存储器161用于存储非暂时性计算机可读指令。具体地，存储器161可以包括一个或多个计算机程序产品，该计算机程序产品可以包括各种形式的计算机可读存储介质，例如易失性存储器和/或非易失性存储器。该易失性存储器例如可以包括随机存取存储器(RAM)和/或高速缓冲存储器(cache)等。该非易失性存储器例如可以包括只读存储器(ROM)、硬盘、闪存等。
该处理器162可以是中央处理单元(CPU)或者具有数据处理能力和/或指令执行能力的其它形式的处理单元，并且可以控制电子设备160中的其它组件以执行期望的功能。在本公开的一个实施例中，该处理器162用于运行该存储器161中存储的该计 算机可读指令，使得该电子设备160执行前述的本公开各实施例的构建图像深度信息的全部或部分步骤。
本领域技术人员应能理解，为了解决如何获得良好用户体验效果的技术问题，本实施例中也可以包括诸如通信总线、接口等公知的结构，这些公知的结构也应包含在本公开的保护范围之内。
有关本实施例的详细说明可以参考前述各实施例中的相应说明，在此不再赘述。
图17是图示根据本公开的实施例的计算机可读存储介质的示意图。如图17所示，根据本公开实施例的计算机可读存储介质170，其上存储有非暂时性计算机可读指令171。当该非暂时性计算机可读指令171由处理器运行时，执行前述的本公开各实施例的构建图像深度信息的全部或部分步骤。
上述计算机可读存储介质170包括但不限于：光存储介质(例如：CD－ROM和DVD)、磁光存储介质(例如：MO)、磁存储介质(例如：磁带或移动硬盘)、具有内置的可重写非易失性存储器的媒体(例如：存储卡)和具有内置ROM的媒体(例如：ROM盒)。
有关本实施例的详细说明可以参考前述各实施例中的相应说明，在此不再赘述。
图18是图示根据本公开实施例的终端设备的硬件结构示意图。如图18所示，该终端180包括上述构建图像深度信息装置实施例。
该终端设备可以以各种形式来实施，本公开中的终端设备可以包括但不限于诸如移动电话、智能电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、导航装置、车载终端设备、车载显示终端、车载电子后视镜等等的移动终端设备以及诸如数字TV、台式计算机等等的固定终端设备。
作为等同替换的实施方式，该终端180还可以包括其他组件。如图7所示，该终端180可以包括电源单元181、无线通信单元182、A/V(音频/视频)输入单元183、用户输入单元184、感测单元185、接口单元186、控制器187、输出单元188和存储单元189等等。图18示出了具有各种组件的终端，但是应理解的是，并不要求实施所有示出的组件，也可以替代地实施更多或更少的组件。
其中，无线通信单元182允许终端180与无线通信系统或网络之间的无线电通信。A/V输入单元183用于接收音频或视频信号。用户输入单元184可以根据用户输入的命令生成键输入数据以控制终端设备的各种操作。感测单元185检测终端180的当前状态、终端180的位置、用户对于终端180的触摸输入的有无、终端180的取向、终端180的加速或减速移动和方向等等，并且生成用于控制终端180的操作的命令或信号。接口单元186用作至少一个外部装置与终端180连接可以通过的接口。输出单元188被构造为以视觉、音频和/或触觉方式提供输出信号。存储单元189可以存储由控制器187执行的处理和控制操作的软件程序等等，或者可以暂时地存储己经输出或将 要输出的数据。存储单元189可以包括至少一种类型的存储介质。而且，终端180可以与通过网络连接执行存储单元189的存储功能的网络存储装置协作。控制器187通常控制终端设备的总体操作。另外，控制器187可以包括用于再现或回放多媒体数据的多媒体模块。控制器187可以执行模式识别处理，以将在触摸屏上执行的手写输入或者图片绘制输入识别为字符或图像。电源单元181在控制器187的控制下接收外部电力或内部电力并且提供操作各元件和组件所需的适当的电力。
本公开提出的构建图像深度信息的各种实施方式可以使用例如计算机软件、硬件或其任何组合的计算机可读介质来实施。对于硬件实施，本公开提出的视频特征的比对方法的各种实施方式可以通过使用特定用途集成电路(ASIC)、数字信号处理器(DSP)、数字信号处理装置(DSPD)、可编程逻辑装置(PLD)、现场可编程门阵列(FPGA)、处理器、控制器、微控制器、微处理器、被设计为执行这里描述的功能的电子单元中的至少一种来实施，在一些情况下，本公开提出的视频特征的比对方法的各种实施方式可以在控制器187中实施。对于软件实施，本公开提出的视频特征的比对方法的各种实施方式可以与允许执行至少一种功能或操作的单独的软件模块来实施。软件代码可以由以任何适当的编程语言编写的软件应用程序(或程序)来实施，软件代码可以存储在存储单元189中并且由控制器187执行。
有关本实施例的详细说明可以参考前述各实施例中的相应说明，在此不再赘述。
以上结合具体实施例描述了本公开的基本原理，但是，需要指出的是，在本公开中提及的优点、优势、效果等仅是示例而非限制，不能认为这些优点、优势、效果等是本公开的各个实施例必须具备的。另外，上述公开的具体细节仅是为了示例的作用和便于理解的作用，而非限制，上述细节并不限制本公开为必须采用上述具体的细节来实现。
在本公开中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序，本公开中涉及的器件、装置、设备、系统的方框图仅作为例示性的例子并且不意图要求或暗示必须按照方框图示出的方式进行连接、布置、配置。如本领域技术人员将认识到的，可以按任意方式连接、布置、配置这些器件、装置、设备、系统。诸如“包括”、“包含”、“具有”等等的词语是开放性词汇，指“包括但不限于”，且可与其互换使用。这里所使用的词汇“或”和“和”指词汇“和/或”，且可与其互换使用，除非上下文明确指示不是如此。这里所使用的词汇“诸如”指词组“诸如但不限于”，且可与其互换使用。
另外，如在此使用的，在以“至少一个”开始的项的列举中使用的“或”指示分离的列举，以便例如“A、B或C的至少一个”的列举意味着A或B或C，或AB或AC或BC，或ABC(即A和B和C)。此外，措辞“示例的”不意味着描述的例子是优选的或者比其他例子更好。
还需要指出的是，在本公开的系统和方法中，各部件或各步骤是可以分解和/或重新组合的。这些分解和/或重新组合应视为本公开的等效方案。
可以不脱离由所附权利要求定义的教导的技术而进行对在此所述的技术的各种改变、替换和更改。此外，本公开的权利要求的范围不限于以上所述的处理、机器、制造、事件的组成、手段、方法和动作的具体方面。可以利用与在此所述的相应方面进行基本相同的功能或者实现基本相同的结果的当前存在的或者稍后要开发的处理、机器、制造、事件的组成、手段、方法或动作。因而，所附权利要求包括在其范围内的这样的处理、机器、制造、事件的组成、手段、方法或动作。
提供所公开的方面的以上描述以使本领域的任何技术人员能够做出或者使用本公开。对这些方面的各种修改对于本领域技术人员而言是非常显而易见的，并且在此定义的一般原理可以应用于其他方面而不脱离本公开的范围。因此，本公开不意图被限制到在此示出的方面，而是按照与在此公开的原理和新颖的特征一致的最宽范围。
为了例示和描述的目的已经给出了以上描述。此外，此描述不意图将本公开的实施例限制到在此公开的形式。尽管以上已经讨论了多个示例方面和实施例，但是本领域技术人员将认识到其某些变型、修改、改变、添加和子组合。

Claims (12)

  1. 一种构建图像深度信息的方法，其特征在于，包括：
    获取与目标对象相关的深度信息；
    标记目标对象预添加深度信息的区域；
    将获取的所述深度信息添加到所述预添加深度信息的区域。
  2. 根据权利要求1所述的构建图像深度信息的方法，其特征在于，所述标记目标对象预添加深度信息的区域，包括：
    分离所述目标对象的前景和背景；
    提取所述目标对象的前景；
    在所述前景中标记预添加深度信息的区域。
  3. 根据权利要求2所述的构建图像深度信息的方法，其特征在于，所述在所述前景信息中标记预添加深度信息的区域，包括：
    提取所述前景中的关键点；
    基于所述关键点对所述前景进行区域划分；
    标记预添加深度信息的区域的关键点。
  4. 根据权利要求2所述的构建图像深度信息的方法，其特征在于，所述在所述前景中标记预添加深度信息的区域，包括：
    提取所述前景的轮廓信息；
    在所述轮廓信息内标记预添加深度信息的区域。
  5. 根据权利要求4所述的构建图像深度信息的方法，其特征在于，在提取所述前景的轮廓信息后，还包括：
    对所述轮廓信息进行平滑处理。
  6. 根据权利要求1所述的构建图像深度信息的方法，其特征在于，所述获取与目标对象相关的深度信息，具体为：
    从深度信息模板中获取与目标对象相关的深度信息。
  7. 根据权利要求6所述的构建图像深度信息的方法，其特征在于，所述将获取的深度信息添加到预添加深度信息的区域，包括：
    提取所述预添加深度信息的区域的关键点，为第一类关键点；
    提取所述深度信息模板的关键点，为第二类关键点；
    基于所述第一类关键点和所述第二类关键点，将所述深度信息模板中的深度信息添加到预添加深度信息的区域。
  8. 根据权利要求7所述的构建图像深度信息的方法，其特征在于，所述基于所述第一类关键点和所述第二类关键点，将所述深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
    基于所述第一类关键点对所述预添加深度信息的区域进行三角剖分，得到至少一个第一区域；
    基于所述第二类关键点对所述深度信息模板进行三角剖分，得到至少一个第二区域；
    将所述至少一个第二区域对应贴合到所述至少一个第一区域。
  9. 根据权利要求7所述的构建图像深度信息的方法，其特征在于，所述基于所述第一类关键点和所述第二类关键点，将所述深度信息模板中的深度信息添加到预添加深度信息的区域，包括：
    计算所述第一类关键点之间的距离，得出第一距离；
    计算所述第二类关键点之间的距离，得出第二距离；
    根据所述第一距离和所述第二距离之间的比值，对所述深度信息模板进行调整；
    将调整后的所述深度信息模板添加到预添加深度信息的区域。
  10. 一种构建图像深度信息的装置，其特征在于，包括：
    获取模块，用于获取与目标对象相关的深度信息；
    区域标记模块，用于标记目标对象预添加深度信息的区域；
    添加模块，用于将获取的所述深度信息添加到所述预添加深度信息的区域。
  11. 一种电子设备，其特征在于，所述电子设备包括：
    至少一个处理器；以及，
    与所述至少一个处理器通信连接的存储器；其中，
    所述存储器存储有能被所述至少一个处理器执行的指令，所述指令被所述至少一个处理器执行，以使所述至少一个处理器能够执行权利要求1-9任一所述的构建图像深度信息的方法。
  12. 一种非暂态计算机可读存储介质，其特征在于，该非暂态计算机可读存储介质存储计算机指令，该计算机指令用于使计算机执行权利要求1-9任一所述的构建图像深度信息的方法。
PCT/CN2019/073070 2018-06-13 2019-01-25 构建图像深度信息的方法及装置 WO2019237744A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201810619716.8 2018-06-13
CN201810619716.8A CN108833881B (zh) 2018-06-13 2018-06-13 构建图像深度信息的方法及装置

Publications (1)

Publication Number Publication Date
WO2019237744A1 true WO2019237744A1 (zh) 2019-12-19

Family

ID=64142418

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/073070 WO2019237744A1 (zh) 2018-06-13 2019-01-25 构建图像深度信息的方法及装置

Country Status (2)

Country Link
CN (1) CN108833881B (zh)
WO (1) WO2019237744A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108833881B (zh) * 2018-06-13 2021-03-23 北京微播视界科技有限公司 构建图像深度信息的方法及装置
CN113256361A (zh) * 2020-02-10 2021-08-13 阿里巴巴集团控股有限公司 商品发布方法及图像处理方法、装置、设备和存储介质
CN116503570B (zh) * 2023-06-29 2023-11-24 聚时科技(深圳)有限公司 图像的三维重建方法及相关装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102903143A (zh) * 2011-07-27 2013-01-30 国际商业机器公司 用于将二维图像三维化的方法和系统
CN105513007A (zh) * 2015-12-11 2016-04-20 惠州Tcl移动通信有限公司 一种基于移动终端的拍照美颜方法、系统及移动终端
CN107833178A (zh) * 2017-11-24 2018-03-23 维沃移动通信有限公司 一种图像处理方法、装置及移动终端
CN108833881A (zh) * 2018-06-13 2018-11-16 北京微播视界科技有限公司 构建图像深度信息的方法及装置

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102903143A (zh) * 2011-07-27 2013-01-30 国际商业机器公司 用于将二维图像三维化的方法和系统
CN105513007A (zh) * 2015-12-11 2016-04-20 惠州Tcl移动通信有限公司 一种基于移动终端的拍照美颜方法、系统及移动终端
CN107833178A (zh) * 2017-11-24 2018-03-23 维沃移动通信有限公司 一种图像处理方法、装置及移动终端
CN108833881A (zh) * 2018-06-13 2018-11-16 北京微播视界科技有限公司 构建图像深度信息的方法及装置

Also Published As

Publication number Publication date
CN108833881B (zh) 2021-03-23
CN108833881A (zh) 2018-11-16

Similar Documents

Publication Publication Date Title
WO2020207191A1 (zh) 虚拟物体被遮挡的区域确定方法、装置及终端设备
CN108764091B (zh) 活体检测方法及装置、电子设备和存储介质
WO2021121236A1 (zh) 一种控制方法、电子设备、计算机可读存储介质、芯片
CN108986016B (zh) 图像美化方法、装置及电子设备
WO2021238325A1 (zh) 一种图像处理方法及装置
US11176355B2 (en) Facial image processing method and apparatus, electronic device and computer readable storage medium
WO2021213067A1 (zh) 物品显示方法、装置、设备及存储介质
WO2019237744A1 (zh) 构建图像深度信息的方法及装置
JP2010510573A (ja) ３次元画像を合成するシステム及び方法
WO2019242271A1 (zh) 图像变形方法、装置及电子设备
JP2011509451A (ja) 画像データのセグメント化
CN110321768A (zh) 用于生成头部相关传递函数滤波器的布置
WO2019237747A1 (zh) 图像裁剪方法、装置、电子设备及计算机可读存储介质
CN109413399B (zh) 使用深度图合成对象的装置及其方法
EP3989591A1 (en) Resource display method, device, apparatus, and storage medium
CN105430269B (zh) 一种应用于移动终端的拍照方法及装置
WO2020038243A1 (zh) 一种视频摘要生成方法、装置、计算设备和存储介质
WO2023169283A1 (zh) 双目立体全景图像的生成方法、装置、设备、存储介质和产品
US11770603B2 (en) Image display method having visual effect of increasing size of target image, mobile terminal, and computer-readable storage medium
CN108921798A (zh) 图像处理的方法、装置及电子设备
CN113822798B (zh) 生成对抗网络训练方法及装置、电子设备和存储介质
WO2020001016A1 (zh) 运动图像生成方法、装置、电子设备及计算机可读存储介质
WO2019237746A1 (zh) 图像合并的方法和装置
US11769466B2 (en) Image display method and apparatus, device, and storage medium
CN112150347A (zh) 从有限的修改后图像集合中学习的图像修改样式

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19819404

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 01.04.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19819404

Country of ref document: EP

Kind code of ref document: A1