WO2020102977A1 - Procédé de traitement d'images et produit associé - Google Patents

Procédé de traitement d'images et produit associé

Info

Publication number
WO2020102977A1
WO2020102977A1 PCT/CN2018/116441 CN2018116441W WO2020102977A1 WO 2020102977 A1 WO2020102977 A1 WO 2020102977A1 CN 2018116441 W CN2018116441 W CN 2018116441W WO 2020102977 A1 WO2020102977 A1 WO 2020102977A1
Authority
WO
WIPO (PCT)
Prior art keywords
rgb
images
silhouette
group
image
Prior art date
Application number
PCT/CN2018/116441
Other languages
English (en)
Chinese (zh)
Inventor
陈岩
Original Assignee
深圳市欢太科技有限公司
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市欢太科技有限公司, Oppo广东移动通信有限公司 filed Critical 深圳市欢太科技有限公司
Priority to CN201880097668.9A priority Critical patent/CN112703534B/zh
Priority to PCT/CN2018/116441 priority patent/WO2020102977A1/fr
Publication of WO2020102977A1 publication Critical patent/WO2020102977A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects

Definitions

  • This application relates to the field of electronic technology, in particular to an image processing method and related products.
  • electronic devices can be used to run somatosensory games.
  • Somatosensory games can be realized through interaction between the user and the electronic device.
  • the electronic device can obtain human body motion videos and analyze the human body motion videos to obtain the corresponding body motion Game operation, so as to control the operation of the game through the game operation, but in the game process, there may be a long data transmission problem, resulting in poor real-time performance of the user's movement and somatosensory games, so how to improve somatosensory games
  • the problem of the efficiency of data transmission in China needs to be solved urgently.
  • the embodiments of the present application provide an image processing method and related products, which can reduce the communication delay between processes by extracting a silhouette profile, and ensure real-time performance of a somatosensory game.
  • an embodiment of the present application provides an image processing method, which is applied to an electronic device, the electronic device includes a depth sensor, and the method includes:
  • the target video of the human body is acquired through the depth sensor, the target video includes a sequence of RGB-D image pairs, and the sequence of RGB-D image pairs includes multiple sets of RGB-D images , Each pair of RGB-D images includes a frame of RGB images and a depth map corresponding to the RGB images;
  • a first silhouette image of the human body is extracted from each group of RGB-D images in the RGB-D image pair sequence by a preset algorithm to obtain multiple first silhouettes Figure, each group of RGB-D images corresponds to a first sketch;
  • An extracting unit configured to extract the first silhouette of the human body from each group of RGB-D images in the RGB-D image pair sequence through a preset algorithm when the first process of the somatosensory game is running, to obtain A plurality of first silhouette images, each group of RGB-D images corresponds to a first silhouette image; and, a silhouette outline is extracted from the plurality of first silhouette images to obtain a plurality of silhouette outline images, each One silhouette map corresponds to one silhouette outline map, and the plurality of silhouette outline maps are used for transmission to the second process of the somatosensory game;
  • the processing unit is configured to perform a recovery operation on the plurality of silhouette contour images when the plurality of silhouette contour images are transmitted to the second process, to obtain a plurality of second silhouette images, each silhouette contour image corresponding to a first Two silhouette figures.
  • an embodiment of the present application provides an electronic device, including a processor, a memory, a communication interface, and one or more programs, wherein the one or more programs are stored in the memory and configured by the above
  • the processor executes, and the above program includes instructions for performing the steps in the first aspect of the embodiments of the present application.
  • an embodiment of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing the computer program, and the computer program is operable to cause the computer to execute as implemented in the present application Examples of some or all of the steps described in the first aspect.
  • the computer program product may be a software installation package.
  • the image processing method and related products described in the embodiments of the present application obtain the target video of the human body through the depth sensor during the process of running the somatosensory game on the electronic device.
  • the target video includes a sequence of RGB-D image pairs.
  • the first silhouette image is extracted from each group of RGB-D images in the sequence of RGB-D image pairs through a preset algorithm to obtain multiple first silhouette images. Extract silhouette silhouettes from multiple silhouette silhouettes.
  • multiple silhouette silhouettes are transferred to the second process, restore multiple silhouette silhouettes to obtain multiple second silhouette silhouettes.
  • the silhouette profile is extracted from the first silhouette in one process, and the second silhouette is restored from the silhouette in the second process.
  • the transmission time of the silhouette from the first process to the second process is less than that from the first
  • the process directly transmits the transmission duration of the first sketch to the second process, thereby reducing the communication delay between the first process and the second process and ensuring the real-time nature of the somatosensory game.
  • FIG. 1A is a schematic structural diagram of an electronic device provided by an embodiment of the present application.
  • FIG. 1B is a schematic flowchart of an image processing method disclosed in an embodiment of the present application.
  • FIG. 2 is a schematic flowchart of another image processing method disclosed in an embodiment of the present application.
  • FIG. 3 is a schematic flowchart of another image processing method disclosed in an embodiment of the present application.
  • FIG. 5 is a schematic structural diagram of an image processing apparatus disclosed in an embodiment of the present application.
  • the electronic devices involved in the embodiments of the present application may include various handheld devices with wireless communication functions, in-vehicle devices, wearable devices, computing devices, or other processing devices connected to a wireless modem, and various forms of user equipment (user equipment, UE), mobile station (MS), terminal device, etc.
  • UE user equipment
  • MS mobile station
  • terminal device etc.
  • the devices mentioned above are collectively referred to as electronic devices.
  • FIG. 1A is a schematic structural diagram of an electronic device disclosed in an embodiment of the present application.
  • the electronic device 100 may include a control circuit, and the control circuit may include a storage and processing circuit 110.
  • the storage and processing circuit 110 may be a memory, such as a hard disk drive memory, a non-volatile memory (such as flash memory or other electronic programmable read-only memory used to form a solid-state drive, etc.), a volatile memory (such as static or dynamic random access memory) Take the memory, etc.), etc., the embodiments of the present application are not limited.
  • the processing circuit in the storage and processing circuit 110 may be used to control the operation of the electronic device 100.
  • the processing circuit can be implemented based on one or more microprocessors, microcontrollers, digital master-slave headphone switching controllers, baseband processors, power management units, audio codec chips, dedicated integrated circuits, display driver integrated circuits, etc. .
  • the storage and processing circuit 110 can be used to run software in the electronic device 100, such as Internet browsing applications, voice over Internet (VOIP) phone call applications, email applications, media playback applications, operating system functions Wait. These software can be used to perform some control operations, such as camera-based image acquisition, ambient light sensor-based ambient light measurement, proximity sensor-based proximity sensor measurement, and information based on status indicators such as light-emitting diode status indicators Display functions, touch event detection based on touch sensors, functions associated with displaying information on multiple (e.g., layered) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals
  • the control operations associated with collecting and processing button press event data and other functions in the electronic device 100 are not limited in the embodiments of the present application.
  • the electronic device 100 may also include an input-output circuit 150.
  • the input-output circuit 150 may be used to enable the electronic device 100 to input and output data, that is, to allow the electronic device 100 to receive data from an external device and also to allow the electronic device 100 to output data from the electronic device 100 to an external device.
  • the input-output circuit 150 may further include a sensor 170.
  • the sensor 170 may include an ambient light sensor, a proximity sensor based on light and capacitance, and a touch sensor (for example, a light-based touch sensor and / or a capacitive touch sensor, where the touch sensor may be part of a touch display screen or may be used as a The touch sensor structure is used independently), acceleration sensor, gravity sensor, and other sensors.
  • the input-output circuit 150 may also include one or more displays, such as the display 130.
  • the display 130 may include a liquid crystal display, an organic light emitting diode display, an electronic ink display, a plasma display, or a display using other display technologies or a combination of several.
  • the display 130 may include a touch sensor array (ie, the display 130 may be a touch display screen).
  • the touch sensor may be a capacitive touch sensor formed by an array of transparent touch sensor electrodes (such as indium tin oxide (ITO) electrodes), or may be a touch sensor formed using other touch technologies, such as sonic touch, pressure sensitive touch, resistance Touch, optical touch, etc. are not limited in the embodiments of the present application.
  • the audio component 140 may be used to provide audio input and output functions for the electronic device 100.
  • the audio component 140 in the electronic device 100 may include a speaker, a microphone, a buzzer, a tone generator, and other components for generating and detecting sound.
  • the communication circuit 120 may be used to provide the electronic device 100 with the ability to communicate with external devices.
  • the communication circuit 120 may include analog and digital input-output interface circuits, and wireless communication circuits based on radio frequency signals and / or optical signals.
  • the wireless communication circuit in the communication circuit 120 may include a radio frequency transceiver circuit, a power amplifier circuit, a low noise amplifier, a switch, a filter, and an antenna.
  • the wireless communication circuit in the communication circuit 120 may include a circuit for supporting near field communication (NFC) by transmitting and receiving near-field coupled electromagnetic signals.
  • the communication circuit 120 may include a near field communication antenna and a near field communication transceiver.
  • the communication circuit 120 may also include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna, and so on.
  • the electronic device 100 may further include a battery, a power management circuit, and other input-output units 160.
  • the input-output unit 160 may include buttons, joysticks, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light emitting diodes, other status indicators, and the like.
  • the user can input commands through the input-output circuit 150 to control the operation of the electronic device 100, and can use the output data of the input-output circuit 150 to realize receiving status information and other outputs from the electronic device 100.
  • FIG. 1B is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • the image processing method described in this embodiment is applied to an electronic device as shown in FIG. 1A.
  • the electronic device includes a depth sensor ,
  • the image processing method includes:
  • the target video includes a sequence of RGB-D image pairs, and the sequence of RGB-D image pairs includes multiple groups of RGB- For D images, each group of RGB-D image pairs includes a frame of RGB images and a depth map corresponding to the RGB images.
  • the embodiment of the present application is applied to a scenario where an electronic device runs a somatosensory game.
  • a target video of a user action can be obtained through a depth sensor, and the sequence of RGB-D image pairs included in the target video is according to the time of video acquisition Arranged in order, the content contained in the target video is the video of the user's coherent action, and each group of RGB-D images in the sequence contains the human posture of the user at the corresponding time in the sequence, where each group of RGB-D images
  • the RGB image in refers to the RGB three-channel color image, the RGB image and the depth map are in one-to-one correspondence, and the pixels in the RGB image and the depth map are also in one-to-one correspondence.
  • each group of RGB-D images corresponds to a first sketch.
  • the first silhouette image of the human body can be extracted from each group of RGB-D images in the RGB-D image pair sequence, so that the game operation corresponding to the user's action can be recognized through the first silhouette image, wherein ,
  • the preset algorithm can be a face detection algorithm, such as a face detection algorithm based on histogram of histogram (HOG), or a face detection algorithm based on sparse code histogram (histograms of sparse codes, HSC) , Not limited here.
  • HOG histogram of histogram
  • HSC sparse code histogram
  • the first process refers to the process of extracting the silhouette image from the target video in the electronic device
  • the second process refers to the process of displaying the silhouette image.
  • the electronic device may run the first process to extract the first image
  • the operation of the sketch and then need to transfer the first sketch to the second process, because the memory space occupied by the first sketch is larger, the communication time required to transfer the first sketch is longer, and the memory space is also reduced
  • the efficiency of reading and writing will affect the real-time nature of the somatosensory game.
  • the silhouette outline is extracted from the first silhouette, the first memory of the first silhouette is larger than the second memory of the silhouette, for example, if the first silhouette
  • the first memory is 14kb, and the second memory of the extracted silhouette profile may be 4-5kb. Transferring the silhouette profile from the first process to the second process will reduce the communication time, improve the read and write efficiency of the memory space, and ensure Real-time nature of somatosensory games.
  • a silhouette outline is extracted from the plurality of first silhouette maps to obtain a plurality of silhouette outline maps, which may include the following steps:
  • each group of RGB-D images in the plurality of groups of RGB-D images Perform a diffusion search on the depth map of the human body to obtain multiple sets of pixels on the edge of the human body.
  • Each set of pixels corresponds to a set of RGB-D images.
  • a collection of pixels generates a silhouette profile, and multiple silhouette profiles are obtained.
  • a set of bone joint points corresponding to each group of RGB-D images can be detected, and the set of bone joint points includes
  • the depth map of the group of RGB-D images can be diffused and searched according to the multiple bone joint points corresponding to each group of RGB-D images to obtain the silhouette outline of the group of RGB-D images, thereby, Multiple silhouette contour maps can be determined according to multiple bone joint point sets corresponding to multiple sets of RGB-D images.
  • the depth map of each group of RGB-D images is subjected to diffusion search to obtain a set of multiple pixel points of the edge contour of the human body, including:
  • A1 Search for the corresponding pixels with the same depth value near each bone joint point in the plurality of bone joint points on the depth map of each group of RGB-D images in the plurality of groups of RGB-D images.
  • A2 Retaining the search termination point in each of the plurality of pixel points corresponding to each of the plurality of bone joint points in the plurality of pixel points corresponding to each group of RGB-D images in the plurality of groups of RGB-D images
  • the pixel points are used as pixel points on the edge contour of the human body to obtain a plurality of pixel point sets of the human edge contour, and each group of RGB-D images corresponds to a pixel point set.
  • each bone joint point in the plurality of bone joint points corresponding to each group of RGB-D images it can be searched according to the diffusion of the bone joint point to find pixels with the same depth value in the surrounding neighborhood, and find For pixels with different depth values, stop searching for the bone joint point, and continue searching for the next bone joint point.
  • the pixel point at the end of the search corresponding to each bone joint point can be used as the pixel point of the human edge contour to obtain Multiple pixel points corresponding to the skeletal joint points, thus, a set of pixel points corresponding to multiple skeletal joint points corresponding to each group of RGB-D images, and multiple pixel points corresponding to multiple groups of RGB-D images can be found set.
  • the game frame rate of the somatosensory game may also be obtained. If the game frame rate is less than a preset threshold, executing the extraction of the silhouette outline from the plurality of first silhouette maps To get multiple silhouettes.
  • the game frame rate of the somatosensory game can be determined. If the game frame rate is less than the preset threshold, it indicates that the real-time performance of the somatosensory game may be poor. Therefore, the extraction from the plurality of first silhouette images may be performed Silhouette profile, the operation to get multiple silhouette profiles.
  • a restoration operation can be performed on each silhouette profile to obtain the restored second silhouette map.
  • the region growth algorithm can be used Restore the silhouette profile.
  • performing a restoration operation on the plurality of silhouette contour images to obtain a plurality of second silhouette images may include the following steps:
  • a silhouette outline map corresponding to each group of RGB-D images in the plurality of groups of RGB-D images through a region growth algorithm and a plurality of bone joint points in each group of RGB-D images in the plurality of groups of RGB-D images Perform a recovery operation to obtain the plurality of second silhouette images.
  • the region growth algorithm can merge pixels with similar properties together. For each region, first specify a seed point as the starting point of growth, and then compare the pixel points and seed points in the neighborhood around the seed point. The points with similar properties are combined and continue to grow outward until no pixels satisfying the conditions are included.
  • multiple bone joint points in each group of RGB-D images can be used as seed points.
  • the region growth algorithm and the multiple bone joint points in each group of RGB-D images in the multiple groups of RGB-D images correspond to each group of RGB-D images in the multiple groups of RGB-D images
  • To perform the recovery operation to obtain the plurality of second silhouette maps which may include the following steps:
  • each bone joint point in the plurality of bone joint points corresponding to each group of RGB-D images as a seed point for diffusion growth to obtain multiple pixel point regions corresponding to the multiple bone joint points, where, Each pixel area includes multiple target pixel points corresponding to each bone joint point;
  • each bone joint point in the plurality of bone joint points corresponding to each group of RGB-D images is used as a seed point to diffuse and grow, to obtain the corresponding Multiple pixel areas can include the following steps:
  • all pixels on the silhouette outline can be traversed to determine whether the traversed pixel is a skeletal joint point, and if so, the pixels corresponding to the skeletal joint point
  • the points are diffused in multiple directions in the neighborhood, for example, they can be diffused and grown in four or eight directions in the neighborhood to obtain the pixel area corresponding to the bone joint point.
  • the following steps may be further included:
  • the second silhouette image can be drawn in the game interface displayed on the electronic device, so that the user can confirm whether his movement is the same as the preset according to the second silhouette image in the game interface The game operation is consistent.
  • the image processing method described in the embodiments of the present application obtains the target video of the human body through the depth sensor during the process of running the somatosensory game on the electronic device.
  • the target video includes a sequence of RGB-D image pairs, which is run in the somatosensory game
  • the first silhouette image is extracted from each group of RGB-D images in the sequence of RGB-D image pairs through a preset algorithm to obtain multiple first silhouette images, which are extracted from multiple first silhouette images Silhouette profile, get multiple silhouette profile images, when multiple silhouette profile images are transferred to the second process, restore the multiple silhouette profile images to obtain multiple second silhouette images, so, you can pass the first process Extract the silhouette outline from the first silhouette, and restore the second silhouette from the silhouette in the second process.
  • the transmission time of the silhouette from the first process to the second process is shorter than that from the first process to the second.
  • the transmission duration of the first sketch is directly transmitted by the two processes, thereby reducing the communication delay between the first process and the second process, and ensuring the real-time nature of the somatosensory game.
  • Each bone joint point set in the key node set includes multiple bone joint points corresponding to each group of RGB-D images.
  • each group of RGB-D images in the plurality of groups of RGB-D images Perform a diffusion search on the depth map of the human body to obtain a set of multiple pixel points of the edge contour of the human body, generate a silhouette outline map according to the set of pixel points corresponding to each group of RGB-D images in the multiple sets of RGB-D images, and obtain multiple silhouette outlines Figure, the multiple silhouette profiles are used for transmission to the second process of the somatosensory game.
  • the image processing method described in the embodiments of the present application obtains the target video of the human body through the depth sensor during the process of running the somatosensory game on the electronic device.
  • the target video includes a sequence of RGB-D image pairs, which is run in the somatosensory game
  • the first silhouette image is extracted from each group of RGB-D images in the sequence of RGB-D image pairs through a preset algorithm to obtain multiple first silhouette images, which are extracted from multiple first silhouette images Silhouette profile, get multiple silhouette profile images, when multiple silhouette profile images are transferred to the second process, restore the multiple silhouette profile images to obtain multiple second silhouette images, and then combine the RGB-D images in the sequence
  • the second silhouette image corresponding to a group of RGB-D images is drawn in the game interface.
  • the silhouette outline image can be extracted from the first silhouette image in the first process, and restored in the second process through the silhouette outline image.
  • the transfer time of the sketch map from the first process to the second process is less than the transfer time of the first sketch map from the first process to the second process, which can reduce the first process and the second process
  • the communication delay between them ensures the real-time nature of somatosensory games.
  • FIG. 3 is a schematic flowchart of another embodiment of an image processing method according to an embodiment of the present application.
  • the image processing method described in this embodiment is applied to the electronic device shown in FIG. 1A.
  • the device includes a depth sensor.
  • the method may include the following steps:
  • a target video of a human body through the depth sensor the target video includes a sequence of RGB-D image pairs, and the sequence of RGB-D image pairs includes multiple sets of RGB- For D images, each group of RGB-D image pairs includes a frame of RGB images and a depth map corresponding to the RGB images.
  • Each bone joint point set in the key node set includes multiple bone joint points corresponding to each group of RGB-D images.
  • each group of RGB-D images in the plurality of groups of RGB-D images Perform a diffusion search on the depth map of the human body to obtain a set of multiple pixel points of the edge contour of the human body, generate a silhouette outline map according to the set of pixel points corresponding to each group of RGB-D images in the multiple sets of RGB-D images, and obtain multiple silhouette outlines Figure, the multiple silhouette profiles are used for transmission to the second process of the somatosensory game.
  • a silhouette corresponding to each group of RGB-D images in the plurality of groups of RGB-D images through a region growth algorithm and a plurality of bone joint points in each group of RGB-D images in the plurality of groups of RGB-D images The contour map performs a recovery operation to obtain the plurality of second sketch maps.
  • the image processing method described in the embodiment of the present application acquires the target video of the human body through the depth sensor during the process of running the somatosensory game on the electronic device.
  • the target video includes a sequence of RGB-D image pairs.
  • a first silhouette image is extracted from each group of RGB-D images in the RGB-D image pair sequence by a preset algorithm to obtain multiple first silhouette images
  • a silhouette outline is extracted from multiple first silhouette images to obtain Multiple silhouette contour maps
  • each of the silhouette contours is obtained by the area growth algorithm and multiple bone joint points in each group of RGB-D images in multiple groups of RGB-D images
  • the image is restored, and multiple second silhouette images are obtained, and the second silhouette image corresponding to each group of RGB-D images in the sequence is drawn in the game interface of the RGB-D image.
  • the silhouette outline is extracted from the first silhouette, and the second silhouette is restored from the silhouette in the second process.
  • the transmission time of the silhouette from the first process to the second process is shorter than that from the first process to the second
  • the process directly transmits the transmission duration of the first sketch, thereby reducing the communication delay between the first process and the second process and ensuring the real-time nature of the somatosensory game.
  • FIG. 4 is an electronic device provided by an embodiment of the present application, including: a processor and a memory; and one or more programs, the one or more programs are stored in the In the memory, and configured to be executed by the processor, the program includes instructions for performing the following steps:
  • a first silhouette image of the human body is extracted from each group of RGB-D images in the RGB-D image pair sequence by a preset algorithm to obtain multiple first silhouettes Figure, each group of RGB-D images corresponds to a first sketch;
  • the first silhouette image of the human body is extracted from each group of RGB-D images in the sequence of RGB-D image pairs through a preset algorithm to obtain multiple first silhouette images
  • the program also includes instructions for performing the following steps:
  • the program further includes instructions for performing the following steps:
  • Each bone joint point set in the set includes multiple bone joint points corresponding to each group of RGB-D images;
  • each group of RGB-D images in the plurality of groups of RGB-D images according to the plurality of bone joint points corresponding to each group of RGB-D images in the plurality of groups of RGB-D images and a preset diffusion determination condition
  • the image is subjected to diffusion search to obtain a plurality of pixel point sets of the edge contour of the human body, each pixel point set corresponds to a group of RGB-D images, according to the pixel points corresponding to each group of RGB-D images in the plurality of groups of RGB-D images
  • the collection generates silhouette silhouette maps to obtain multiple silhouette silhouette maps.
  • the multiple sets of RGB-D images are determined based on multiple bone joint points corresponding to each of the multiple sets of RGB-D images and preset diffusion determination conditions.
  • the program includes instructions for performing the following steps:
  • the program in performing the restoring operation on the plurality of silhouette contour images to obtain a plurality of second silhouette images, includes instructions for performing the following steps:
  • a silhouette outline map corresponding to each group of RGB-D images in the plurality of groups of RGB-D images through a region growth algorithm and a plurality of bone joint points in each group of RGB-D images in the plurality of groups of RGB-D images Perform a recovery operation to obtain the plurality of second silhouette images.
  • the program further includes instructions for performing the following steps:
  • Each bone joint point in the plurality of bone joint points corresponding to each group of RGB-D images is used as a seed point for diffusion growth to obtain multiple pixel point regions corresponding to the multiple bone joint points, wherein each The pixel area includes multiple target pixel points corresponding to each bone joint point;
  • the second silhouette map corresponding to each group of RGB-D images is generated according to all target pixels in the multiple target pixel areas in the area of the multiple pixel areas that are greater than the preset area.
  • each bone joint point is used as a seed point to diffuse and grow, and the corresponding multiple bone joint points are obtained.
  • the program includes instructions for performing the following steps:
  • the multiple pixel points are diffused as multiple seed points in multiple directions in the neighborhood to obtain multiple pixel point regions corresponding to the multiple seed points, wherein each seed point corresponds to a pixel point region, so When each pixel grows to a black pixel, the growth is terminated.
  • the program further includes instructions for performing the following steps:
  • the program after transmitting the first application data packet according to the first target number of retransmissions, the program further includes instructions for performing the following steps:
  • FIG. 5 is a schematic structural diagram of an image processing apparatus provided by this embodiment.
  • the image processing device is applied to an electronic device as shown in FIG. 1A, the electronic device includes a depth sensor, and the image processing device includes an acquisition unit 501, an extraction unit 502, and a processing unit 503, wherein,
  • the acquiring unit 501 is configured to acquire a target video of a human body through the depth sensor during a somatosensory game of the electronic device, the target video includes a sequence of RGB-D image pairs, and the pair of RGB-D images
  • the sequence includes multiple groups of RGB-D images, and each pair of RGB-D images includes a frame of RGB images and a depth map corresponding to the RGB images;
  • the extracting unit 502 is configured to extract the first silhouette of the human body from each group of RGB-D images in the RGB-D image pair sequence through a preset algorithm when the somatosensory game runs the first process Figures, a plurality of first silhouette maps are obtained, and each group of RGB-D images corresponds to a first silhouette map; and, silhouette outlines are extracted from the plurality of first silhouette maps to obtain a plurality of silhouette outline maps, Each first silhouette map corresponds to a silhouette outline map, and the plurality of silhouette outline maps are used for transmission to the second process of the somatosensory game;
  • the processing unit 503 is configured to perform a recovery operation on the plurality of silhouette contour images when the plurality of silhouette contour images are transmitted to the second process, to obtain a plurality of second silhouette images, each silhouette contour image Corresponds to a second sketch.
  • the The extraction unit is specifically used for:
  • the extracting unit is specifically configured to:
  • Each bone joint point set in the set includes multiple bone joint points corresponding to each group of RGB-D images;
  • each group of RGB-D images in the plurality of groups of RGB-D images according to the plurality of bone joint points corresponding to each group of RGB-D images in the plurality of groups of RGB-D images and a preset diffusion determination condition
  • the image is subjected to diffusion search to obtain a plurality of pixel point sets of the edge contour of the human body, each pixel point set corresponds to a group of RGB-D images, according to the pixel points corresponding to each group of RGB-D images in the plurality of groups of RGB-D images
  • the collection generates silhouette silhouette maps to obtain multiple silhouette silhouette maps.
  • each of the plurality of sets of RGB-D images according to the plurality of bone joint points corresponding to each of the plurality of sets of RGB-D images and a preset diffusion determination condition The depth map of a group of RGB-D images performs diffusion search to obtain a collection of multiple pixel points of the edge contour of the human body.
  • the extraction unit is specifically used to:
  • processing unit is specifically used to:
  • a silhouette outline map corresponding to each group of RGB-D images in the plurality of groups of RGB-D images through a region growth algorithm and a plurality of bone joint points in each group of RGB-D images in the plurality of groups of RGB-D images Perform a recovery operation to obtain the plurality of second silhouette images.
  • each region of the RGB-D images in the plurality of groups of RGB-D images is paired with a plurality of skeletal joint points in the RGB-D images of each group in the plurality of groups of RGB-D images through a region growth algorithm With respect to the corresponding silhouette outline image, a restoration operation is performed to obtain the plurality of second silhouette image aspects.
  • the processing unit is specifically configured to:
  • Each bone joint point in the plurality of bone joint points corresponding to each group of RGB-D images is used as a seed point to diffuse and grow, to obtain a plurality of pixel point regions corresponding to the multiple bone joint points, wherein, each A pixel area includes multiple target pixel points corresponding to each bone joint point;
  • the second silhouette map corresponding to each group of RGB-D images is generated according to all target pixels in the multiple target pixel areas in the area of the multiple pixel areas that are greater than the preset area.
  • each bone joint point is used as a seed point for diffusion growth, to obtain multiple pixel points corresponding to the multiple bone joint points Regionally, the processing unit is specifically used to:
  • the multiple pixel points are diffused as multiple seed points in multiple directions in the neighborhood to obtain multiple pixel point regions corresponding to the multiple seed points, wherein each seed point corresponds to a pixel point region, so When each pixel grows to a black pixel, the growth is terminated.
  • the processing unit is further configured to:
  • the acquiring unit 501 is further configured to acquire the game frame rate of the somatosensory game. If the game frame rate is less than a preset threshold, the extracting unit executes the Extract the silhouette outline from the picture, and get the operation of multiple silhouette outline pictures.
  • the image processing device described in the embodiments of the present application acquires the target video of the human body through the depth sensor during the process of running the somatosensory game on the electronic device, the target video includes the sequence of RGB-D image pairs, and runs in the somatosensory game
  • the first silhouette image is extracted from each group of RGB-D images in the sequence of RGB-D image pairs through a preset algorithm to obtain multiple first silhouette images, which are extracted from multiple first silhouette images Silhouette profile, get multiple silhouette profile images, when multiple silhouette profile images are transferred to the second process, restore the multiple silhouette profile images to obtain multiple second silhouette images, so, you can pass the first process Extract the silhouette outline from the first silhouette, and restore the second silhouette from the silhouette in the second process.
  • the transmission time of the silhouette from the first process to the second process is shorter than that from the first process to the second.
  • the transmission duration of the first sketch is directly transmitted by the two processes, thereby reducing the communication delay between the first process and the second process, and ensuring the real-time nature of the somatosensory game.
  • An embodiment of the present application also provides a computer storage medium, wherein the computer storage medium stores a computer program for electronic data exchange, and the computer program causes the computer to execute part of any image processing method described in the above method embodiments Or all steps.
  • An embodiment of the present application further provides a computer program product, the computer program product includes a non-transitory computer-readable storage medium storing a computer program, the computer program is operable to cause the computer to execute as described in the above method embodiments Some or all steps of any image processing method.
  • the disclosed device may be implemented in other ways.
  • the device embodiments described above are only schematic.
  • the division of the unit is only a logical function division.
  • there may be another division manner for example, multiple units or components may be combined or may Integration into another system, or some features can be ignored, or not implemented.
  • the displayed or discussed mutual couplings or direct couplings or communication connections may be indirect couplings or communication connections through some interfaces, devices or units, and may be in electrical or other forms.
  • the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.
  • the above integrated unit can be implemented in the form of hardware or software program modules.
  • the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it may be stored in a computer-readable memory.
  • the technical solution of the present application essentially or part of the contribution to the existing technology or all or part of the technical solution can be embodied in the form of a software product, the computer software product is stored in a memory, Several instructions are included to enable a computer device (which may be a personal computer, server, network device, etc.) to perform all or part of the steps of the methods described in the embodiments of the present application.
  • the foregoing memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disk and other media that can store program codes.
  • the program may be stored in a computer-readable memory, and the memory may include: a flash disk , ROM, RAM, magnetic disk or optical disk, etc.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

L'invention concerne un procédé de traitement d'images et un produit associé. Le procédé selon l'invention consiste : lorsqu'un dispositif électronique exécute un jeu de détection de mouvement, à acquérir une vidéo cible d'un corps humain au moyen d'un capteur de profondeur, la vidéo cible comprenant une séquence de paires d'images RVB-D ; lorsque le jeu de détection de mouvement est exécuté dans un premier processus, à extraire une première image de silhouette de chaque groupe d'images RVB-D dans la séquence de paires d'images RVB-D afin d'obtenir des premières images de silhouette multiples ; à extraire des contours de silhouette à partir des premières images de silhouette multiples afin d'obtenir des images de contours de silhouette multiples ; et, lorsque les images de contours de silhouette multiples sont transmises à un deuxième processus, à effectuer une opération de récupération sur les images de contours de silhouette multiples afin d'obtenir des deuxièmes images de silhouette multiples. Ainsi, la durée de transmission des images de contours de silhouette du premier processus au deuxième processus est inférieure à la durée de transmission directe des premières images de silhouette du premier processus au deuxième processus, de sorte que le retard de communication entre le premier et le deuxième processus puisse être réduit, ce qui garantit les performances en temps réel du jeu de détection de mouvement.
PCT/CN2018/116441 2018-11-20 2018-11-20 Procédé de traitement d'images et produit associé WO2020102977A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN201880097668.9A CN112703534B (zh) 2018-11-20 2018-11-20 图像处理方法及相关产品
PCT/CN2018/116441 WO2020102977A1 (fr) 2018-11-20 2018-11-20 Procédé de traitement d'images et produit associé

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2018/116441 WO2020102977A1 (fr) 2018-11-20 2018-11-20 Procédé de traitement d'images et produit associé

Publications (1)

Publication Number Publication Date
WO2020102977A1 true WO2020102977A1 (fr) 2020-05-28

Family

ID=70773105

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2018/116441 WO2020102977A1 (fr) 2018-11-20 2018-11-20 Procédé de traitement d'images et produit associé

Country Status (2)

Country Link
CN (1) CN112703534B (fr)
WO (1) WO2020102977A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537121A (zh) * 2021-07-28 2021-10-22 浙江大华技术股份有限公司 身份识别方法和装置、存储介质及电子设备
CN114648542A (zh) * 2022-03-11 2022-06-21 联宝(合肥)电子科技有限公司 一种目标物提取方法、装置、设备及可读存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116152051B (zh) * 2023-02-27 2023-12-01 上海福柯斯智能科技有限公司 一种x射线图像的分块配准图像剪影方法和装置
CN116309940B (zh) * 2023-03-22 2023-11-24 浪潮智慧科技有限公司 一种基于动画弹窗组件的地图信息显示方法、设备及介质

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1629888A (zh) * 2003-12-17 2005-06-22 中国科学院自动化研究所 一种骨架化物体重建方法
CN101354792A (zh) * 2008-09-08 2009-01-28 北京航空航天大学 一种保留模型细节特征的轮廓线远程绘制方法
KR20140060807A (ko) * 2012-11-12 2014-05-21 한남대학교 산학협력단 사용자 신체 및 음성을 이용한 컴퓨터 인터페이스 방법
WO2016035941A1 (fr) * 2014-09-05 2016-03-10 동국대학교 산학협력단 Système et procédé de reconnaissance de pose utilisant des données spatiales 3d sur un modèle humain
CN105786016A (zh) * 2016-03-31 2016-07-20 深圳奥比中光科技有限公司 无人机以及rgbd图像的处理方法
CN107194964A (zh) * 2017-05-24 2017-09-22 电子科技大学 一种基于实时人体三维重建的vr社交系统及其方法
CN107945269A (zh) * 2017-12-26 2018-04-20 清华大学 基于多视点视频的复杂动态人体对象三维重建方法及系统
CN108154551A (zh) * 2017-11-29 2018-06-12 深圳奥比中光科技有限公司 实时动态重建三维人体模型的方法及系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8638985B2 (en) * 2009-05-01 2014-01-28 Microsoft Corporation Human body pose estimation
CN102609683B (zh) * 2012-01-13 2014-02-05 北京邮电大学 一种基于单目视频的人体关节自动标注方法
US9813666B2 (en) * 2012-05-29 2017-11-07 Qualcomm Incorporated Video transmission and reconstruction
CN103916431A (zh) * 2013-01-04 2014-07-09 云联（北京）信息技术有限公司 一种人机交互系统及方法
CN103679175B (zh) * 2013-12-13 2017-02-15 电子科技大学 一种基于深度摄像机的快速3d骨骼模型检测方法

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1629888A (zh) * 2003-12-17 2005-06-22 中国科学院自动化研究所 一种骨架化物体重建方法
CN101354792A (zh) * 2008-09-08 2009-01-28 北京航空航天大学 一种保留模型细节特征的轮廓线远程绘制方法
KR20140060807A (ko) * 2012-11-12 2014-05-21 한남대학교 산학협력단 사용자 신체 및 음성을 이용한 컴퓨터 인터페이스 방법
WO2016035941A1 (fr) * 2014-09-05 2016-03-10 동국대학교 산학협력단 Système et procédé de reconnaissance de pose utilisant des données spatiales 3d sur un modèle humain
CN105786016A (zh) * 2016-03-31 2016-07-20 深圳奥比中光科技有限公司 无人机以及rgbd图像的处理方法
CN107194964A (zh) * 2017-05-24 2017-09-22 电子科技大学 一种基于实时人体三维重建的vr社交系统及其方法
CN108154551A (zh) * 2017-11-29 2018-06-12 深圳奥比中光科技有限公司 实时动态重建三维人体模型的方法及系统
CN107945269A (zh) * 2017-12-26 2018-04-20 清华大学 基于多视点视频的复杂动态人体对象三维重建方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113537121A (zh) * 2021-07-28 2021-10-22 浙江大华技术股份有限公司 身份识别方法和装置、存储介质及电子设备
CN114648542A (zh) * 2022-03-11 2022-06-21 联宝(合肥)电子科技有限公司 一种目标物提取方法、装置、设备及可读存储介质

Also Published As

Publication number Publication date
CN112703534A (zh) 2021-04-23
CN112703534B (zh) 2024-04-02

Similar Documents

Publication Publication Date Title
WO2020102977A1 (fr) Procédé de traitement d'images et produit associé
WO2021036536A1 (fr) Procédé de filmage vidéo et dispositif électronique
WO2018103525A1 (fr) Procédé et dispositif de suivi de point clé facial, et support de stockage
US10769464B2 (en) Facial recognition method and related product
US9779527B2 (en) Method, terminal device and storage medium for processing image
US11340777B2 (en) Method for editing text and mobile terminal
WO2017054605A1 (fr) Procédé et dispositif de traitement d'image
CN108038825B (zh) 一种图像处理方法及移动终端
CN104765446A (zh) 电子设备和控制电子设备的方法
CN104954631B (zh) 一种视频处理方法、装置和系统
CN109005336B (zh) 一种图像拍摄方法及终端设备
CN107885448B (zh) 应用触摸操作的控制方法、移动终端及可读存储介质
WO2021109931A1 (fr) Procédé de commande tactile et dispositif vestimentaire
US10528845B2 (en) Method, device, and storage medium for generating GIF file
CN108600544B (zh) 一种单手控制方法及终端
US11895069B2 (en) Message sending method and mobile terminal
WO2017088434A1 (fr) Procédé et appareil d'apprentissage matriciel de modèle de visage humain ainsi que support d'enregistrement
WO2021190387A1 (fr) Procédé de sortie de résultat de détection, dispositif électronique et support
CN108108081A (zh) 基于双面屏的信息显示方法、移动终端及可读存储介质
CN108536349B (zh) 一种图标管理方法及移动终端
CN111158815B (zh) 一种动态壁纸模糊方法、终端和计算机可读存储介质
CN105513098B (zh) 一种图像处理的方法和装置
CN111556248A (zh) 拍摄方法、装置、存储介质及移动终端
CN108829600B (zh) 算法库的测试方法、装置、存储介质和电子设备
CN111027406A (zh) 图片识别方法、装置、存储介质及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18941024

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18941024

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 29.09.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 18941024

Country of ref document: EP

Kind code of ref document: A1