WO2023015868A1 - 图像背景生成方法及装置、计算机可读存储介质 - Google Patents

图像背景生成方法及装置、计算机可读存储介质 Download PDF

Info

Publication number
WO2023015868A1
WO2023015868A1 PCT/CN2022/078656 CN2022078656W WO2023015868A1 WO 2023015868 A1 WO2023015868 A1 WO 2023015868A1 CN 2022078656 W CN2022078656 W CN 2022078656W WO 2023015868 A1 WO2023015868 A1 WO 2023015868A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
background
camera
processing device
image processing
Prior art date
Application number
PCT/CN2022/078656
Other languages
English (en)
French (fr)
Inventor
王敏波
陈显义
Original Assignee
华为技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 华为技术有限公司 filed Critical 华为技术有限公司
Publication of WO2023015868A1 publication Critical patent/WO2023015868A1/zh

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/16Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformation in the plane of the image
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/50Image enhancement or restoration by the use of more than one image, e.g. averaging, subtraction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/70Determining position or orientation of objects or cameras
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/80Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration

Definitions

  • the present application relates to the technical field of image processing, in particular to a method and device for generating an image background, and a computer-readable storage medium.
  • human body imaging is extracted from video conference images and then fused into a pre-configured virtual background to obtain a synthetic image.
  • the present application provides a method and device for generating an image background, and a computer-readable storage medium, which can solve the current problem of poor imaging effect of a composite image caused by a mismatch between human body imaging and a virtual background.
  • a method for generating an image background includes: the image processing device acquires description information of a simulated scene corresponding to a target scene, where the description information includes a three-dimensional scene model.
  • the image processing device acquires camera parameters of a camera deployed in a target scene.
  • the image processing device uses the camera parameters to perform projection transformation on the three-dimensional scene model to obtain the image background corresponding to the camera.
  • the image processing device uses the camera parameters of the camera to perform projection transformation on the 3D scene model of the simulation scene, and the image under the perspective of the camera can be obtained Background, the image background is consistent with the viewing angle and visible area of the real image captured by the camera. Therefore, in the synthesized image obtained by replacing the background of the real image collected by the camera with the background of the image, the foreground and the background always match, and the imaging effect of the synthesized image is better.
  • the solution of the present application is not limited to the deployment position of the camera and the posture of the camera, and has high flexibility.
  • an implementation manner in which the image processing device obtains camera parameters of a camera deployed in the target scene includes: the image processing device determines camera extrinsic parameters of the camera according to the camera's pose in the world coordinate system.
  • the image processing device determines camera parameters according to the camera extrinsic parameters and the acquired camera intrinsic parameters of the camera.
  • camera parameters are obtained based on camera intrinsic parameters and camera extrinsic parameters.
  • the image processing device may use a calibration algorithm to calibrate to obtain the internal camera parameters.
  • the description information of the simulated scene also includes pose information corresponding to one or more reference position points in the simulated scene, and the pose information includes the horizontal position of the reference position point in the world coordinate system and/or the reference position point corresponding camera angle.
  • the image processing device displays a top view of the simulated scene, and displays reference position points on the top view.
  • the image processing device receives a selection instruction for a target reference position point among one or more reference position points, the image processing device determines the pose of the camera in the world coordinate system according to the pose information corresponding to the target reference position point.
  • the above-mentioned one or more reference position points are pre-set position points where the camera may be placed, and the present application does not limit the setting position and setting quantity of the reference position points.
  • the pose information corresponding to the reference position point includes the horizontal position of the reference position point in the world coordinate system.
  • the image processing device determines the position and orientation of the camera in the world coordinate system according to the pose information corresponding to the target reference position point
  • the implementation process includes: when the image processing device receives a selection instruction for the target reference position point, the image processing device displays the target reference position area, and the target reference position area includes the target reference position point and one or Multiple optional location points.
  • the image processing device When the image processing device receives a confirmation instruction for a target optional position point among one or more selectable position points, the image processing device, according to the horizontal position of the target reference position point in the world coordinate system and the distance between the target optional position point and The relative position of the target reference position point, determine the horizontal position of the target optional position point in the world coordinate system, and use the horizontal position of the target optional position point in the world coordinate system as the horizontal position of the camera in the world coordinate system; or , when the image processing device receives the confirmation instruction for the target position reference point, the image processing device uses the horizontal position of the target position reference point in the world coordinate system as the horizontal position of the camera in the world coordinate system.
  • the image processing device when the user selects a reference position point on the display interface of the image processing device, the image processing device can center on the selected reference position point and radiate other position points near the selected reference position point, through The way of UI interaction realizes the user's adjustability to the determined camera position.
  • the image processing device may also display the image background corresponding to the target reference position point.
  • the image processing device can display the image background corresponding to the reference position point, so that the user can intuitively adjust the selection based on the image background corresponding to the reference position point. camera position.
  • multiple markers are set in the target scene, and the spatial positions of the multiple markers in the world coordinate system are stored in the image processing device.
  • Another implementation manner in which the image processing device obtains the camera parameters of the camera deployed in the target scene includes: the image processing device obtains a target image, and the target image is obtained by shooting the target scene by the camera. The image processing device determines camera parameters according to the imaging positions of the multiple markers in the target image and the spatial positions of the multiple markers in the world coordinate system.
  • the image processing device automatically determines the camera parameters according to the spatial positions of multiple landmarks in the world coordinate system and the imaging positions in the target image, without requiring the user to manually select the deployment position of the camera, which can improve user experience.
  • the image processing equipment can calculate the changed camera parameters by itself based on the re-captured images of the camera and the spatial positions of the landmarks, making the operation more convenient. good.
  • the image processing device displays an image background and a background adjustment control, where the background adjustment control is used to adjust the background direction, background position and/or background visible area.
  • the image processing device adjusts the image background according to the first manipulation instruction.
  • the image processing device displays the image background and background adjustment controls for the image background, and realizes the adjustment of the image background through UI interaction, so that the adjusted image background can better match the camera angle of view.
  • the three-dimensional scene model includes a first background model and a second background model, and the second background model is located in a space formed by the first background model.
  • the image processing device uses the camera parameters to projectively transform the 3D scene model to obtain the image background corresponding to the camera, including: the image processing device uses the camera parameters to projectively transform the first background model to obtain the first sub-image background.
  • the image processing device uses the camera parameters to perform projection transformation on the second background model to obtain the background of the second sub-image.
  • the image processing device superimposes the background of the second sub-image on the background of the first sub-image to obtain the background of the image.
  • the image processing device may also display the image background and an adjustment control for the background of the second sub-image, where the adjustment control is used to adjust the position and/or size of the background of the second sub-image.
  • the image processing device adjusts the background of the second sub-image according to the second manipulation instruction.
  • superimposing the background of the second sub-image on the background of the first sub-image refers to superimposing the layer containing the background of the second sub-image on the layer containing the background of the first sub-image, that is, the background of the second sub-image It is in a different layer from the background of the first sub-image, so that the background of the second sub-image can be adjusted independently, which improves the flexibility of adjusting the background of the image.
  • the image processing device acquires the foreground imaging in the first image, and the first image is acquired by a camera.
  • the image processing device fuses the foreground imaging and the image background to obtain the second image.
  • the image processing device acquires the foreground imaging in the first image, including: the image processing device performs human body semantic segmentation on the first image to obtain a human body mask in the first image.
  • the image processing device obtains the foreground imaging according to the human body mask and the first image.
  • I is the pixel matrix representing the second image
  • is the human body mask
  • F is the pixel matrix representing the first image
  • B is the pixel matrix representing the image background.
  • an image processing device in a second aspect, includes multiple functional modules, and the multiple functional modules interact to implement the methods in the above first aspect and various implementation manners thereof.
  • the multiple functional modules can be implemented based on software, hardware or a combination of software and hardware, and the multiple functional modules can be combined or divided arbitrarily based on specific implementations.
  • an image processing device including: a processor and a memory;
  • the memory is used to store a computer program, and the computer program includes program instructions
  • the processor is configured to invoke the computer program to implement the methods in the above first aspect and various implementation manners thereof.
  • a computer-readable storage medium In a fourth aspect, a computer-readable storage medium is provided. Instructions are stored on the computer-readable storage medium. When the instructions are executed by a processor, the above-mentioned first aspect and the methods in each implementation manner thereof are implemented.
  • a chip is provided, and the chip includes a programmable logic circuit and/or program instructions, and when the chip is running, implements the methods in the above first aspect and various implementation manners thereof.
  • FIG. 1 is a schematic diagram of an application scenario involved in a method for generating an image background provided by an embodiment of the present application
  • FIG. 2 is a schematic flowchart of a method for generating an image background provided by an embodiment of the present application
  • FIG. 3 is a schematic diagram of a simulation scenario provided by an embodiment of the present application.
  • Fig. 4 is a schematic diagram of converting a three-dimensional scene model of a simulated scene into a top view according to an embodiment of the present application
  • FIG. 5 is a schematic representation of a camera angle provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a display interface of an image processing device provided in an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application.
  • Fig. 8 is a schematic diagram of a first sub-image background provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a second sub-image background provided by an embodiment of the present application.
  • FIG. 10 is a schematic diagram of an image background provided by an embodiment of the present application.
  • Fig. 11 is a schematic diagram of a generation process of a foreground image in a first image provided by an embodiment of the present application
  • Fig. 12 is a schematic diagram of a second image provided by an embodiment of the present application.
  • Fig. 13 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application.
  • Fig. 14 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application.
  • Fig. 15 is a schematic structural diagram of an image processing device provided by an embodiment of the present application.
  • Fig. 16 is a schematic structural diagram of another image processing device provided by an embodiment of the present application.
  • Fig. 17 is a schematic structural diagram of another image processing device provided by an embodiment of the present application.
  • Fig. 18 is a schematic structural diagram of another image processing device provided by the embodiment of the present application.
  • Fig. 19 is a schematic structural diagram of another image processing device provided by an embodiment of the present application.
  • Fig. 20 is a block diagram of an image processing device provided by an embodiment of the present application.
  • Image coordinate system is a coordinate system with the upper left vertex of the image captured by the camera as the coordinate origin.
  • the x-axis (horizontal axis) and y-axis (vertical axis) of the image coordinate system are respectively the width and height directions of the image collected by the camera.
  • the camera coordinate system is a three-dimensional rectangular coordinate system established with the optical center of the camera as the origin and the principal optical axis as the z-axis.
  • the x-axis of the camera coordinate system is parallel to the x-axis of the image coordinate system corresponding to the image collected by the camera.
  • the y-axis of the camera coordinate system is parallel to the y-axis of the image coordinate system corresponding to the image captured by the camera.
  • the world coordinate system can describe the position of the camera in the real world, and can also describe the position of objects in the image captured by the camera in the real world. Based on the pose of the camera in the world coordinate system, the camera coordinate system corresponding to the camera can be converted to the world coordinate system.
  • Pose The pose of the camera in the world coordinate system is used to describe the position and attitude of the camera.
  • the position refers to the coordinates of the camera in the world coordinate system, which can be represented by a translation matrix.
  • Pose refers to the orientation of the camera in the world coordinate system, which can be represented by a rotation matrix.
  • the extrinsic parameters of the camera can be calculated based on the pose of the camera in the world coordinate system.
  • Camera parameters can be represented by a camera matrix.
  • the camera parameters may be calculated based on camera intrinsic parameters and camera extrinsic parameters.
  • the internal parameters of the camera can be expressed as an internal parameter matrix K:
  • f x and f y represent the focal length of the camera, which is related to the pixel size.
  • c x represents the offset of the main optical axis of the camera in the x-axis direction of the image coordinate system corresponding to the image collected by the camera, that is, c x is the abscissa of the optical center of the camera in the image coordinate system corresponding to the image collected by the camera Coordinates, usually half the width of the image captured by this camera.
  • c y represents the offset of the main optical axis of the camera on the y-axis of the image coordinate system corresponding to the image collected by the camera, that is, c y is the vertical axis of the optical center of the camera in the image coordinate system corresponding to the image collected by the camera Coordinates, usually half of the height of the image captured by the camera.
  • R is the rotation matrix, which is a matrix with 3 rows and 3 columns, which is used to represent the pose of the camera.
  • t is the translation matrix, which is a matrix with 3 rows and 1 column, which is used to represent the position of the camera.
  • the camera parameters may be calculated according to the spatial positions of the multiple three-dimensional points in the world coordinate system and the corresponding imaging positions of the multiple three-dimensional points in the images collected by the camera.
  • W represents the camera matrix
  • i in formula (5) is from 0 to N
  • N is the number of multiple three-dimensional points, N>2
  • the formula is solved by singular value decomposition (SVD) to solve the least squares problem (5) Solve to get the camera matrix W.
  • SVD singular value decomposition
  • Projective transformation Use the camera parameters of a camera to perform projective transformation on a 3D point in the world coordinate system, and obtain the corresponding pixel of the 3D point in the image coordinate system corresponding to the image collected by the camera.
  • the pre-configured virtual background is directly used to replace the background in the video conference image captured by the camera without considering the shooting position of the camera, there may be a problem that the human body imaging does not match the virtual background, resulting in poor imaging effect of the composite image. Difference.
  • the embodiment of the present application provides a method for generating an image background.
  • the image processing device uses the camera parameters of the camera to analyze the 3D scene of the simulation scene.
  • the model performs projection transformation to obtain the image background under the perspective of the camera, which is consistent with the perspective and visible area of the real image collected by the camera. Therefore, in the synthesized image obtained by replacing the background of the real image collected by the camera with the background of the image, the foreground and the background always match, and the imaging effect of the synthesized image is better.
  • FIG. 1 is a schematic diagram of an application scenario involved in a method for generating an image background provided by an embodiment of the present application.
  • the application scenario includes multiple conference terminals. Each conference terminal can communicate with all other conference terminals.
  • the application scenario shown in Figure 1 is taken as an example including two conference terminals (conference terminals 1-2), and the actual scenario may also include three, four or more conference terminals. The number of terminals is not limited.
  • the conference terminal is integrated with a camera, or the conference terminal is connected to the camera.
  • the video stream collected by the camera integrated in the conference terminal and the video stream collected by the camera connected to the conference terminal are collectively referred to as the video stream collected by the conference terminal.
  • the conference terminal 1 is used to send the video stream 1 collected by the conference terminal 1 to the conference terminal 2
  • the conference terminal 2 is used to send the video stream 2 collected by the conference terminal 2 to the conference terminal 1, so as to realize the conference terminal A video conference between a participant on side 1 and a participant on side 2 of the conference terminal.
  • Conference endpoints are used to display images or play video streams.
  • the conference terminal is an electronic device with a display function such as a large screen, an electronic whiteboard, a mobile phone, a tablet computer, or a smart wearable device.
  • the conference terminal also has an image processing function, that is, the conference terminal can be used as an image processing device.
  • the conference terminal is connected to a device with image processing functions such as a server, a server cluster or a cloud computing center (not shown in the figure), the device with image processing function is used as an image processing device, and the conference terminal is used as an image display device.
  • the image background generation method provided by the embodiment of the present application is not only applicable to the video conference scene, but also can be applied to other video communication scenes such as video chat, and can also be applied to other scenes that require image background replacement.
  • the embodiment of the present application applies to the application scene The type of is not limited.
  • FIG. 2 is a schematic flowchart of a method for generating an image background provided by an embodiment of the present application. As shown in FIG. 2 , the method includes the following steps 201 to 205 .
  • Step 201 The image processing device acquires description information of a simulated scene corresponding to a target scene, where the description information includes a three-dimensional scene model.
  • the image processing device is a conference terminal with image processing capability or a device with image processing capability connected to the conference terminal.
  • the embodiment of the present application is described by taking an image processing device as an example of a conference terminal.
  • the image processing device is a conference terminal in an application scenario as shown in FIG. 1 .
  • the target scene is a real scene
  • the simulated scene is a virtual three-dimensional scene constructed by simulating the target scene.
  • the target scene is a meeting room
  • the simulated scene is a virtual meeting room constructed to simulate the meeting room.
  • FIG. 3 is a schematic diagram of a simulation scenario provided by an embodiment of the present application.
  • the simulated scene includes walls, ground and tables.
  • the 3D scene model of the simulated scene includes the color of the 3D points forming the simulated scene and the coordinates of the 3D points in the world coordinate system.
  • the corresponding relationship between the target scene and one or more simulated scenes is pre-stored in the image processing device.
  • the image processing device may display top views of the multiple simulated scenes for the user to select, and then acquire description information of the simulated scene selected by the user.
  • the top view of the simulated scene may be obtained by projectively transforming the three-dimensional scene model of the simulated scene by using the camera parameters when the camera is shot down.
  • FIG. 4 is a schematic diagram of converting a three-dimensional scene model of a simulated scene into a top view according to an embodiment of the present application.
  • the camera parameters of the camera are used to project the 3D scene model shown in the left figure in Figure 4 Transformation, the top view shown in the right figure in Figure 4 can be obtained.
  • Step 202 the image processing device acquires camera parameters of cameras deployed in the target scene.
  • the implementation process of step 202 includes: the image processing device determines the camera extrinsic parameters of the camera according to the pose of the camera in the world coordinate system.
  • the image processing device determines camera parameters of the camera according to the camera extrinsic parameters and the acquired camera intrinsic parameters of the camera.
  • the image processing device uses a calibration algorithm to calibrate the internal parameters of the camera, and this embodiment of the present application does not limit the type of the calibration algorithm used.
  • the description information of the simulated scene includes pose information corresponding to one or more reference position points in the simulated scene.
  • the one or more reference position points are pre-set position points where the camera may be placed, and the embodiment of the present application does not limit the set position and set quantity of the reference position points.
  • the pose information of the reference position point includes the horizontal position of the reference position point in the world coordinate system and/or the camera angle corresponding to the reference position point. Among them, the camera angle is used to represent the camera pose.
  • the horizontal position of the reference position point in the world coordinate system may be represented by horizontal two-dimensional coordinates (x, y) in the world coordinate system.
  • the camera angle corresponding to the reference position point can be represented in the form of (roll, pitch, yaw), where roll represents the rotation angle, pitch represents the pitch angle, and yaw represents the yaw angle.
  • FIG. 5 is a schematic representation of a camera angle provided by an embodiment of the present application.
  • the implementation of determining the pose of the camera in the world coordinate system by the image processing device includes the following steps 2021 and 2022 .
  • step 2021 the image processing device displays a top view of the simulated scene, and displays reference position points on the top view.
  • the pose information of the reference location point includes a horizontal position of the reference location point in the world coordinate system and a camera angle corresponding to the reference location point.
  • FIG. 6 is a schematic diagram of a display interface of an image processing device provided in an embodiment of the present application. As shown in Fig. 6, there are 3 reference position points shown on the top view, marked as A, B and C.
  • a plane coordinate system is established with O as the origin, and the plane coordinate system is the coordinate system of the world coordinate system on the horizontal plane, that is, both the x-axis and the y-axis are located on the horizontal plane.
  • the coordinates of the reference position point A in the plane coordinate system are (2, 2), and the camera angle corresponding to the reference position point A is (0°, 0°, -45°); the reference position point B is in the plane coordinate system
  • the coordinates of the reference position point B are (2, 5), and the camera angle corresponding to the reference position point B is (0°, 0°, 45°);
  • the coordinates of the reference position point C in the plane coordinate system are (10, 3), and the reference position point
  • the camera angle corresponding to point C is (0°, 0°, 180°).
  • the coordinate unit of the horizontal position is meter. Since the plane coordinate system is the coordinate system of the world coordinate system on the horizontal plane, the coordinates of the reference position point in the plane coordinate system are also the horizontal position of the reference position point in the world coordinate system.
  • step 2022 when the image processing device receives a selection instruction for a target reference position point among the one or more reference position points, the image processing device determines that the camera is in the world according to the pose information corresponding to the target reference position point.
  • the pose in the coordinate system when the image processing device receives a selection instruction for a target reference position point among the one or more reference position points, the image processing device determines that the camera is in the world according to the pose information corresponding to the target reference position point. The pose in the coordinate system.
  • the pose information of the reference position point includes the horizontal position of the reference position point in the world coordinate system.
  • the implementation of step 2022 includes the following steps S221 to S223.
  • step S221 when the image processing device receives a selection instruction for the target reference position point, the image processing device displays the target reference position area, the target reference position area includes the target reference position point and a or multiple optional location points.
  • FIG. 7 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application.
  • the image processing device when it receives a selection instruction for the reference position point C in Figure 6, it can display a reference position area with the reference position point C as the center, including the reference position point C and a number of optional location points (represented by grid vertices).
  • the image processing device may center on the selected reference position point and radiate other position points near the selected reference position point , by means of user interface (user interface, UI) interaction, to realize the user's adjustability to the determined camera position.
  • user interface user interface
  • the image processing device displays the image background corresponding to the target reference position point.
  • the image background corresponding to the target reference position point refers to an image obtained by projecting the simulated scene under the camera angle of view corresponding to the target reference position point.
  • the image processing device can use the pose information of the target reference position point as part of the pose information of the camera, and combine the camera installation height to determine the camera extrinsic parameters, and then use the camera intrinsic parameters and camera extrinsic parameters to perform projection transformation on the 3D scene model to obtain the target
  • the image background corresponding to the reference position point For example, please continue to refer to FIG. 7 , the image background corresponding to the reference position point C is also displayed on the display interface of the image processing device.
  • the image processing device when the user selects a reference position point on the display interface of the image processing device, the image processing device can display the image background corresponding to the reference position point, so that the user can intuitively view the reference position point based on the image background corresponding to the reference position point Adjust the selected camera position.
  • step S222 when the image processing device receives the confirmation instruction for the target position reference point, the image processing device uses the horizontal position of the target position reference point in the world coordinate system as the horizontal position of the camera in the world coordinate system.
  • step S223 when the image processing device receives the confirmation instruction for the target optional position point among the one or more optional position points, the image processing device will use the horizontal position of the target reference position point in the world coordinate system and The relative position of the target optional position point and the target reference position point, determine the horizontal position of the target optional position point in the world coordinate system, and use the horizontal position of the target optional position point in the world coordinate system as the camera in the world coordinate system lower horizontal position.
  • the optional location located on the right side of the reference location point C and adjacent to the reference location point C The horizontal position of a point in the world coordinate system can be expressed as (10.5, 3). If the image processing device receives a confirmation instruction for the optional location point, it determines that the horizontal position of the camera deployed in the target scene in the world coordinate system is (10.5, 3).
  • the terminal may also center on the selected optional location point and continue to radiate other location points near the selected optional location point, Repeat this until the user determines the location point.
  • the trigger condition of the selection instruction is different from the trigger condition of the confirmation instruction.
  • the image processing device detects a click operation on a certain position point
  • the image processing device determines that a selection instruction for the position point is received.
  • the image processing device detects a double-click operation on a certain position point
  • the image processing device determines that a confirmation instruction for the position point is received.
  • the image processing device detects a click operation on a certain location point
  • the image processing device determines that a selection instruction for the location point has been received; when the image processing device detects a click operation on the location point again , the image processing device determines that a confirmation instruction for the position point is received.
  • the above steps S221 to S223 can also be implemented as an alternative: when the image processing device receives the selection instruction for the target reference position point, the image processing device directly places the target reference position point in the world coordinate system The horizontal position is determined as the horizontal position of the camera in the world coordinate system.
  • the image processing device acquires the horizontal position of the camera in the world coordinate system.
  • the image processing device may initialize the height of the camera in the world coordinate system. For example, if the image processing device is a large screen and the camera is integrated on the top of the large screen, the image processing device may initialize the installation height of the camera to the top height of the large screen.
  • the image processing device may also prompt the user to input the installation height of the camera, and support the function of manually changing the installation height of the camera. If the image processing device receives the input camera installation height, the image processing device takes the input camera installation height as the height of the camera in the world coordinate system.
  • the image processing device may obtain a translation matrix corresponding to the camera based on the horizontal position of the camera in the world coordinate system and the height of the camera in the world coordinate system.
  • the pose information of the reference location point includes a camera angle corresponding to the reference location point.
  • the implementation of step 2022 includes: when the image processing device receives the selection instruction for the target reference position point, the image processing device calculates the rotation matrix corresponding to the camera deployed in the target scene based on the camera angle corresponding to the target reference position point.
  • the camera parameters (ie, the camera matrix) can be calculated based on the above formula (3).
  • multiple markers are set in the target scene, and the image processing device stores the spatial positions of the multiple markers in the world coordinate system.
  • the implementation process of step 202 includes: the image processing device acquires a target image, and the target image is captured by a camera on a target scene.
  • the image processing device determines camera parameters according to the imaging positions of the multiple markers in the target image and the spatial positions of the multiple markers in the world coordinate system.
  • the spatial positions of multiple anchor points in the simulation scene may be pre-configured.
  • the image processing device may prompt the user to respectively set markers at multiple anchor points in the target scene.
  • the image processing device may display the simulated scene and each anchor point in the simulated scene, so that the user can intuitively obtain the desired setting position of the marker.
  • the user can also set multiple markers in the target scene by himself, and input the spatial positions of the multiple markers to the image processing device.
  • the camera can be enabled to shoot the target scene.
  • the image processing device acquires the image captured by the camera on the target scene, it can use an image recognition algorithm to identify the imaging of the marker in the image, and use the center of the imaging of the marker as the imaging position of the marker. Further, according to the imaging positions of the multiple markers in the image and the spatial positions of the multiple markers, the camera parameters (that is, the camera matrix) are calculated based on the above formula (5).
  • the image processing device automatically determines the camera parameters according to the spatial positions of multiple landmarks in the world coordinate system and the imaging positions in the target image, without requiring the user to manually select the deployment position of the camera, which can improve user experience.
  • the image processing device can calculate the changed Camera parameters, better operation convenience.
  • Step 203 the image processing device uses the camera parameters to perform projection transformation on the three-dimensional scene model to obtain the image background corresponding to the camera.
  • the image processing device uses the camera parameters of the camera to perform projection transformation on the 3D scene model of the simulation scene, and can obtain the The image background is consistent with the viewing angle and visible area of the real image collected by the camera. Therefore, in the synthetic image obtained after replacing the background of the real image collected by the camera with the background of the image, the foreground and background always match, and the imaging effect of the synthetic image is better.
  • the three-dimensional scene model of the simulated scene includes a first background model and a second background model.
  • the second background model is located in the space formed by the first background model.
  • the first background model includes walls, ground and so on.
  • the second background model includes tables, chairs, and the like.
  • the implementation process of step 203 includes the following steps 2031 to 2033.
  • step 2031 the image processing device uses camera parameters to perform projective transformation on the first background model to obtain the first sub-image background.
  • the first background model includes the wall and the ground in the simulated scene shown in FIG. 3 , and if the deployment position of the camera in the target scene corresponds to the reference position point C shown in FIG.
  • the image processing device performs projective transformation on the first background model by using the camera parameters, so as to obtain the background of the first sub-image as shown in FIG. 8 .
  • step 2032 the image processing device uses the camera parameters to perform projective transformation on the second background model to obtain the background of the second sub-image.
  • the second background model includes the table in the simulated scene shown in FIG. 3, and if the deployment position of the camera in the target scene corresponds to the reference position point C shown in FIG.
  • the camera parameters are used to perform projective transformation on the second background model, and the background of the second sub-image as shown in FIG. 9 can be obtained.
  • step 2033 the image processing device superimposes the background of the second sub-image on the background of the first sub-image to obtain the background of the image.
  • Overlaying the background of the second sub-image on the background of the first sub-image refers to superimposing the layer containing the background of the second sub-image on the layer containing the background of the first sub-image, that is, the background of the second sub-image and the background of the first sub-image
  • the background of the image is in a different layer, so that the background of the second sub-image can be individually adjusted.
  • the image processing device superimposes the background of the second sub-image shown in FIG. 9 on the background of the first sub-image shown in FIG. 8 to obtain the background of the image shown in FIG. 10 .
  • the 3D scene model of the simulated scene is an integral model
  • the image processing device may use camera parameters to perform integral projection transformation on the 3D scene model to obtain an image background corresponding to the camera.
  • the image processing device may also perform the following steps 204 and 205 after obtaining the image background corresponding to the camera.
  • Step 204 the image processing device acquires the foreground image in the first image, and the first image is acquired by the camera.
  • the first image may be any image collected by the target camera after the target camera is fixedly deployed in the target scene.
  • step 204 includes: the image processing device performs human body semantic segmentation on the first image to obtain a human body mask in the first image. Then the image processing device obtains the foreground image according to the human body mask and the first image.
  • FIG. 11 is a schematic diagram of a generation process of a foreground image in a first image provided by an embodiment of the present application.
  • Step 205 the image processing device fuses the foreground imaging in the first image and the image background corresponding to the camera to obtain a second image.
  • the second image is a composite image.
  • I is the pixel matrix representing the second image
  • is the human body mask
  • F is the pixel matrix representing the first image
  • B is the pixel matrix representing the image background.
  • the second image shown in FIG. 12 can be obtained by fusing the foreground imaging shown in FIG. 11 with the image background shown in FIG. 10 .
  • the conference terminal may further send the second image to a remote conference terminal, so as to implement video conference communication.
  • the image processing device uses the camera parameters of the camera to analyze the three-dimensional image of the simulation scene.
  • the scene model performs projection transformation to obtain the image background under the camera's perspective, which is consistent with the perspective and visible area of the real image collected by the camera. Therefore, in the synthesized image obtained by replacing the background of the real image collected by the camera with the background of the image, the foreground and the background always match, and the imaging effect of the synthesized image is better.
  • the solution of this application is not limited to the deployment position of the camera and the posture of the camera, and has high flexibility.
  • the image processing device may center on the selected reference position point and radiate For other position points near the selected reference position point, the user can adjust the determined camera position through UI interaction.
  • the image processing device may also adjust the image background through UI interaction.
  • the image processing device may fuse the foreground imaging in the first image and the adjusted image background to obtain the second image.
  • the image processing device may adjust the image background as a whole, and the specific implementation process includes the following steps S11 to S12:
  • step S11 the image processing device displays the image background and background adjustment controls.
  • the background adjustment controls are used to adjust the background orientation, background position and/or background visible area.
  • the adjustment of the background direction can be realized by adjusting the rotation matrix in the camera extrinsic parameters.
  • the adjustment of the background position can be realized by adjusting the translation matrix in the camera extrinsic parameters.
  • the adjustment of the visible area of the background can be realized by adjusting the focal length of the camera. Adjusting the background direction and adjusting the background position are actually adjusting the background viewing angle.
  • the background adjustment control can also be used to adjust the size of the background, and the size adjustment of the image background can be realized by scaling the image background.
  • the image processing device may acquire the foreground imaging in the image captured by the camera deployed in the target scene, and superimpose the foreground imaging on the background of the obtained image to obtain a pre-synthesized image.
  • the image processing device displays the precomposed image and background adjustment controls for the background of the image in the precomposed image.
  • FIG. 13 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application. As shown in FIG. 13 , a pre-synthesized image and a background editing interface are displayed on the image processing device, and the background editing interface includes a background direction adjustment control, a background position adjustment control and a background visible area adjustment control.
  • the background direction adjustment control is represented by rotation axes x, y, and z, corresponding to pitch angle, yaw angle, and rotation angle, respectively.
  • the background position adjustment control is represented by a small square whose position can be moved up and down, left and right.
  • the background visible area adjustment control is represented by a circle with adjustable radius, increasing the radius of the circle means zooming in on the background, and decreasing the radius of the circle means zooming out the background.
  • step S12 in response to the first manipulation instruction on the background adjustment control, the image processing device adjusts the image background according to the first manipulation instruction.
  • the image processing device adjusts the rotation matrix in the camera extrinsic parameters according to the manipulation command to obtain new camera parameters, and then executes the above step 203 again to realize the background direction adjustment.
  • the image processing device adjusts the translation matrix in the camera extrinsic parameters according to the manipulation command to obtain new camera parameters, and then executes the above step 203 again, thereby realizing the adjustment of the background position.
  • the image processing device in response to the manipulation instruction of the background visible area adjustment control, adjusts the focal length of the camera according to the manipulation instruction to adjust the internal parameters of the camera to obtain new camera parameters, and then executes the above step 203 again, so as to realize the visualization of the background.
  • Regional regulation in response to the manipulation instruction of the background visible area adjustment control, the image processing device adjusts the focal length of the camera according to the manipulation instruction to adjust the internal parameters of the camera to obtain new camera parameters, and then executes the above step 203 again, so as to realize the visualization of the background.
  • the image processing device displays the image background and the background adjustment control for the image background, and realizes the overall adjustment of the image background through UI interaction.
  • the image processing device can display the image background in the form of a pre-synthesized image, so that the user can adjust the image background with reference to the foreground imaging in the pre-synthesized image, so that the adjusted image background can better match the camera angle of view, thereby enabling The adjusted image background can better match the foreground in the real image captured by the camera.
  • the image processing device can locally adjust the background of the image, and the specific implementation process includes the following steps S21 to S22:
  • step S21 the image processing device displays the image background and adjustment controls for the second sub-image background in the image background.
  • the adjustment controls for the background of the second sub-image are used to adjust the position and/or size of the background of the second sub-image.
  • the image processing device may acquire the foreground imaging in the image captured by the camera deployed in the target scene, and superimpose the foreground imaging on the background of the obtained image to obtain a pre-synthesized image.
  • the image processing device displays the precomposed image and adjustment controls for the background of the second sub-image in the precomposed image.
  • the background of the second sub-image is a table image
  • FIG. 14 is a schematic diagram of a display interface of another image processing device provided in an embodiment of the present application. As shown in FIG. 14 , the image processing device displays a pre-synthesized image and an operation axis for table imaging. By moving the operation axis, the position of the table image can be moved, and by zooming the operation axis, the table image can be zoomed.
  • step S22 in response to a second manipulation instruction on the adjustment control, the image processing device adjusts the background of the second sub-image according to the second manipulation instruction.
  • the image processing device displays the image background and adjustment controls for sub-image backgrounds in the image background, and realizes partial adjustment of the image background through UI interaction, which improves the flexibility of adjusting the image background.
  • the image processing device can display the image background in the form of a pre-synthesized image, so that the user can locally adjust the image background with reference to the foreground imaging in the pre-synthesized image, so that the adjusted image background can better match the real image captured by the camera. The foreground in the image.
  • the image processing device may also simultaneously display the image background, the background adjustment control, and the adjustment control for the second sub-image background in the image background on the same interface, which is not limited in this embodiment of the present application .
  • the image processing device can store the camera parameters corresponding to the adjusted image background and/or the adjusted image background, so that the image processing device can directly use the stored camera parameters or image background , reducing the probability of the user adjusting the image background again, thereby improving user experience.
  • the implementation process for the image processing device to acquire the image background may include the following steps S31 to S34.
  • step S31 the image processing device displays the first option, the second option and the third option.
  • the first option is used to indicate to initialize the camera parameters
  • the second option is used to indicate to use the camera parameters stored by the image processing device
  • the third option is used to indicate to use the image background stored by the image processing device.
  • step S32 if the image processing device receives a selection instruction for the first option, the image processing device executes the above steps 201 to 203 to obtain an image background.
  • step S33 if the image processing device receives a selection instruction for the second option, the image processing device acquires the stored camera parameters and executes the above step 201, and then uses the stored camera parameters to perform projection transformation on the 3D scene model of the simulated scene , to get the image background.
  • step S34 if the image processing device receives a selection instruction for the third option, the image processing device acquires and displays the stored image background, and then determines the final used image background according to the user's selection.
  • the order of the steps in the method for generating the image background provided in the embodiment of the present application can be adjusted appropriately, and the steps can also be increased or decreased according to the situation. Any person familiar with the technical field within the technical scope disclosed in this application can easily think of changing methods, which should be covered within the scope of protection of this application.
  • the above-mentioned embodiments of the present application are all described by taking the image processing device with a display function as an example.
  • the above method can also be completed by the cooperation of the image processing device and the image display device.
  • the image processing device is used to perform image processing actions.
  • the image display device is used to perform an image display action.
  • FIG. 15 is a schematic structural diagram of an image processing device provided by an embodiment of the present application. As shown in Figure 15, the image processing device 1500 includes:
  • the first obtaining module 1501 is configured to obtain description information of a simulation scene corresponding to a target scene, where the description information includes a three-dimensional scene model.
  • the second acquiring module 1502 is configured to acquire camera parameters of cameras deployed in the target scene.
  • the projective transformation module 1503 is configured to perform projective transformation on the 3D scene model by using the camera parameters to obtain the image background corresponding to the camera.
  • the second acquiring module 1502 is configured to determine the camera extrinsics of the camera according to the pose of the camera in the world coordinate system. And determine camera parameters according to the camera extrinsic parameters and the obtained camera intrinsic parameters of the camera.
  • the description information also includes pose information corresponding to one or more reference position points in the simulation scene, where the pose information includes the horizontal position of the reference position point in the world coordinate system and/or the camera corresponding to the reference position point angle.
  • the image processing device further includes: a display module 1504, configured to display a top view of the simulated scene, and display reference position points on the top view.
  • the determination module 1505 is configured to determine the position of the camera in the world coordinate system according to the pose information corresponding to the target reference position point when the image processing device receives a selection instruction for a target reference position point in one or more reference position points. pose.
  • the pose information includes the horizontal position of the reference position point in the world coordinate system.
  • the display module 1504 is further configured to display a target reference position area when the image processing device receives a selection instruction for the target reference position point, the target reference position area includes the target reference position point and one or more optional location points.
  • the determining module 1505 is configured to, when the image processing device receives a confirmation instruction for a target optional location point in one or more optional location points, according to the horizontal position of the target reference location point in the world coordinate system and the target optional location point The relative position of the position point and the target reference position point, determine the horizontal position of the target optional position point in the world coordinate system, and use the horizontal position of the target optional position point in the world coordinate system as the level of the camera in the world coordinate system Location.
  • multiple markers are set in the target scene, and the spatial positions of the multiple markers in the world coordinate system are stored in the image processing device.
  • the second obtaining module 1502 is configured to: obtain a target image, and the target image is captured by a camera on a target scene.
  • the camera parameters are determined according to the imaging positions of the multiple markers in the target image and the spatial positions of the multiple markers in the world coordinate system.
  • the image processing device 1500 further includes: a display module 1504, configured to display the image background and a background adjustment control, the background adjustment control is used to adjust the background direction, background position and/or background visible area.
  • the first background adjustment module 1506 is configured to adjust the image background according to the first manipulation instruction in response to the first manipulation instruction on the background adjustment control.
  • the three-dimensional scene model includes a first background model and a second background model, and the second background model is located in a space formed by the first background model.
  • Projective transformation module 1503, configured to: use camera parameters to perform projective transformation on the first background model to obtain the first sub-image background; use camera parameters to perform projective transformation on the second background model to obtain the second sub-image background; The image background is superimposed on the first sub-image background to obtain the image background.
  • the image processing device 1500 further includes: a display module 1504 for displaying the image background and adjustment controls for the second sub-image background, the adjustment controls for adjusting the position and/or size of the second sub-image background.
  • the second background adjustment module 1507 is configured to adjust the background of the second sub-image according to the second manipulation instruction in response to the second manipulation instruction on the adjustment control.
  • the image processing device 1500 further includes: a third acquiring module 1508, configured to acquire the foreground imaging in the first image, where the first image is acquired by a camera.
  • An image fusion module 1509 configured to fuse the foreground imaging and the image background to obtain a second image.
  • Fig. 20 is a block diagram of an image processing device provided by an embodiment of the present application.
  • the image processing device may be a conference terminal or a device connected to the conference terminal with an image processing function.
  • the conference terminal can be an electronic device with a display function such as a large screen, an electronic whiteboard, a mobile phone, a tablet computer, or a smart wearable device.
  • the device with an image processing function connected to the conference terminal can be a server, or it can be composed of several servers. composed of server clusters, or a cloud computing center.
  • the image processing device 2000 includes: a processor 2001 and a memory 2002 .
  • the memory 2002 is used for storing computer programs, and the computer programs include program instructions;
  • the processor 2001 is configured to call the computer program to implement the method shown in FIG. 2 .
  • the image processing device 2000 further includes a communication bus 2003 and a communication interface 2004 .
  • the processor 2001 includes one or more processing cores, and the processor 2001 executes various functional applications and data processing by running computer programs.
  • the memory 2002 can be used to store computer programs.
  • the memory may store an operating system and application program units required for at least one function.
  • the operating system can be an operating system such as a real-time operating system (Real Time eXecutive, RTX), LINUX, UNIX, WINDOWS or OS X.
  • the communication interfaces 2004 are used to communicate with other storage devices or network devices.
  • the communication interface of the conference terminal may be used to send an image or video stream to another conference terminal.
  • Network devices can be switches or routers, etc.
  • the memory 2002 and the communication interface 2004 are respectively connected to the processor 2001 through the communication bus 2003 .
  • the embodiment of the present application also provides a computer-readable storage medium, where instructions are stored on the computer-readable storage medium, and when the instructions are executed by a processor, the method for generating an image background as shown in FIG. 2 is realized.
  • the program can be stored in a computer-readable storage medium.
  • the above-mentioned The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, and the like.

Abstract

本申请公开了一种图像背景生成方法及装置、计算机可读存储介质,属于图像处理技术领域。图像处理设备首先获取目标场景对应的模拟场景的描述信息以及部署在目标场景中的相机的相机参数,然后采用该相机的相机参数对描述信息中的三维场景模型进行投影变换,得到该相机对应的图像背景。该图像背景是相机视角下的图像,与相机采集的真实图像的视角和可视区域都是一致的,采用该图像背景更换该相机采集的真实图像中的背景后得到的合成图像中,前景与背景始终都是匹配的,合成图像的成像效果较好。

Description

图像背景生成方法及装置、计算机可读存储介质
本申请要求于2021年08月13日提交的申请号为202110932990.2、发明名称为“图像背景生成方法及装置、计算机可读存储介质”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域,特别涉及一种图像背景生成方法及装置、计算机可读存储介质。
背景技术
在现代会议系统中,随着跨地域的协作越来越多以及成本的要求,对视频会议的需求日益增加。出于对用户所处环境隐私的保护,通常需要将视频会议图像中的真实背景替换为虚拟背景。
在一些研究中,通过提取视频会议图像中的人体成像,然后将人体成像融合至预先配置的虚拟背景中,以得到合成图像。
但是,合成图像中可能会出现人体成像与虚拟背景不匹配的问题,导致合成图像的成像效果较差。
发明内容
本申请提供了一种图像背景生成方法及装置、计算机可读存储介质,可以解决目前由于人体成像与虚拟背景不匹配导致的合成图像的成像效果较差的问题。
第一方面,提供了一种图像背景生成方法。该方法包括:图像处理设备获取目标场景对应的模拟场景的描述信息,该描述信息包括三维场景模型。图像处理设备获取部署在目标场景中的相机的相机参数。图像处理设备采用相机参数对三维场景模型进行投影变换,得到该相机对应的图像背景。
本申请中,通过预先配置目标场景对应的模拟场景,在目标场景中部署相机之后,图像处理设备采用该相机的相机参数对模拟场景的三维场景模型进行投影变换,可以得到该相机视角下的图像背景,该图像背景与相机采集的真实图像的视角和可视区域都是一致的。因此,采用该图像背景更换该相机采集的真实图像中的背景后得到的合成图像中,前景与背景始终都是匹配的,合成图像的成像效果较好。另外,本申请方案不受限于相机的部署位置以及相机的姿态,灵活性较高。
可选地,图像处理设备获取部署在目标场景中的相机的相机参数的一种实现方式,包括:图像处理设备根据相机在世界坐标系下的位姿,确定该相机的相机外参。图像处理设备根据该相机外参以及获取的该相机的相机内参,确定相机参数。该实现方式中,相机参数基于相机内参和相机外参得到。可选地,图像处理设备可以采用标定算法标定得到相机内参。
可选地,模拟场景的描述信息还包括模拟场景中的一个或多个参考位置点对应的位姿信 息,该位姿信息包括参考位置点在世界坐标系下的水平位置和/或参考位置点对应的相机角度。图像处理设备显示模拟场景的俯视图,并在俯视图上显示参考位置点。当图像处理设备接收到针对一个或多个参考位置点中的目标参考位置点的选择指令时,图像处理设备根据目标参考位置点对应的位姿信息,确定相机在世界坐标系下的位姿。
其中,上述一个或多个参考位置点是预先设置的可能会放置相机的位置点,本申请对参考位置点的设置位置和设置数量均不做限定。
可选地,参考位置点对应的位姿信息包括参考位置点在世界坐标系下的水平位置。当图像处理设备接收到针对一个或多个参考位置点中的目标参考位置点的选择指令时,图像处理设备根据目标参考位置点对应的位姿信息,确定相机在世界坐标系下的位姿的实现过程,包括:当图像处理设备接收到针对目标参考位置点的选择指令时,图像处理设备显示目标参考位置区域,目标参考位置区域内包括目标参考位置点以及位于目标参考位置点周围的一个或多个可选位置点。当图像处理设备接收到针对一个或多个可选位置点中的目标可选位置点的确认指令时,图像处理设备根据目标参考位置点在世界坐标系下的水平位置以及目标可选位置点与目标参考位置点的相对位置,确定目标可选位置点在世界坐标系下的水平位置,并将目标可选位置点在世界坐标系下的水平位置作为相机在世界坐标系下的水平位置;或者,当图像处理设备接收到针对目标位置参考点的确认指令时,图像处理设备将目标位置参考点在世界坐标系下的水平位置作为相机在世界坐标系下的水平位置。
本申请中,当用户在图像处理设备的显示界面上选择一个参考位置点时,图像处理设备可以以被选择的参考位置点为中心,辐射出被选择的参考位置点附近的其它位置点,通过UI交互的方式,实现用户对确定的相机位置的可调性。
可选地,当图像处理设备接收到针对目标参考位置点的选择指令时,图像处理设备还可以显示目标参考位置点对应的图像背景。
本申请中,当用户在图像处理设备的显示界面上选择一个参考位置点时,图像处理设备可以显示该参考位置点对应的图像背景,便于用户基于参考位置点对应的图像背景,直观地调整选择的相机位置。
可选地,目标场景中设置有多个标志物,图像处理设备中存储有多个标志物在世界坐标系下的空间位置。图像处理设备获取部署在目标场景中的相机的相机参数的另一种实现方式,包括:图像处理设备获取目标图像,该目标图像由相机对目标场景拍摄得到。图像处理设备根据多个标志物在目标图像中的成像位置以及该多个标志物在世界坐标系下的空间位置,确定相机参数。
本实现方式中,图像处理设备根据多个标志物在世界坐标系下的空间位置以及在目标图像中的成像位置自动确定相机参数,而无需用户手动选择相机的部署位置,可提升用户体验。另外,当相机在目标场景中的位姿发生变化时,无需重新设置标志物,图像处理设备可根据相机重新拍摄的图像以及标志物的空间位置自行计算出变更后的相机参数,操作便捷性更好。
可选地,图像处理设备显示图像背景以及背景调节控件,背景调节控件用于调节背景方向、背景位置和/或背景可视区域。响应于对背景调节控件的第一操控指令,图像处理设备根据第一操控指令调节图像背景。
本申请中,图像处理设备显示图像背景以及针对图像背景的背景调节控件,通过UI交互的方式实现图像背景的可调节,以使调整后的图像背景能够更好地匹配相机视角。
可选地,三维场景模型包括第一背景模型和第二背景模型,第二背景模型位于第一背景模型所形成的空间内。图像处理设备采用相机参数对三维场景模型进行投影变换,得到相机对应的图像背景的实现过程,包括:图像处理设备采用相机参数对第一背景模型进行投影变换,得到第一子图像背景。图像处理设备采用相机参数对第二背景模型进行投影变换,得到第二子图像背景。图像处理设备将第二子图像背景叠加在第一子图像背景上,得到图像背景。进一步地,图像处理设备还可以显示图像背景以及针对第二子图像背景的调节控件,该调节控件用于调节第二子图像背景的位置和/或大小。响应于对调节控件的第二操控指令,图像处理设备根据第二操控指令调节第二子图像背景。
本申请中,将第二子图像背景叠加在第一子图像背景上,是指将包含第二子图像背景的图层叠加在包含第一子图像背景的图层上,即第二子图像背景和第一子图像背景在不同图层中,这样可以实现对第二子图像背景的单独调节,提高了对图像背景的调节灵活性。
可选地,图像处理设备获取第一图像中的前景成像,第一图像由相机采集得到。图像处理设备对前景成像以及图像背景进行融合,得到第二图像。
可选地,图像处理设备获取第一图像中的前景成像的实现方式,包括:图像处理设备对第一图像进行人体语义分割,得到第一图像中的人体掩膜。图像处理设备根据人体掩膜以及第一图像,得到前景成像。
可选地,第一图像、第二图像以及图像背景满足公式:I=αF+(1-α)B。其中,I为表征第二图像的像素矩阵,α为人体掩膜,F为表征第一图像的像素矩阵,B为表征图像背景的像素矩阵。
第二方面,提供了一种图像处理设备。所述图像处理设备包括多个功能模块,所述多个功能模块相互作用,实现上述第一方面及其各实施方式中的方法。所述多个功能模块可以基于软件、硬件或软件和硬件的结合实现,且所述多个功能模块可以基于具体实现进行任意组合或分割。
第三方面,提供了一种图像处理设备,包括:处理器和存储器;
所述存储器,用于存储计算机程序,所述计算机程序包括程序指令;
所述处理器,用于调用所述计算机程序,实现上述第一方面及其各实施方式中的方法。
第四方面,提供了一种计算机可读存储介质,所述计算机可读存储介质上存储有指令,当所述指令被处理器执行时,实现上述第一方面及其各实施方式中的方法。
第五方面,提供了一种芯片,芯片包括可编程逻辑电路和/或程序指令,当芯片运行时,实现上述第一方面及其各实施方式中的方法。
附图说明
图1是本申请实施例提供的一种图像背景生成方法涉及的应用场景示意图;
图2是本申请实施例提供的一种图像背景生成方法的流程示意图;
图3是本申请实施例提供的一种模拟场景的示意图;
图4是本申请实施例提供的一种模拟场景的三维场景模型变换为俯视图的示意图;
图5是本申请实施例提供的一种相机角度的表示示意图;
图6是本申请实施例提供的一种图像处理设备的显示界面示意图;
图7是本申请实施例提供的另一种图像处理设备的显示界面示意图;
图8是本申请实施例提供的一种第一子图像背景的示意图;
图9是本申请实施例提供的一种第二子图像背景的示意图;
图10是本申请实施例提供的一种图像背景的示意图;
图11是本申请实施例提供的一种第一图像中的前景成像的生成过程示意图;
图12是本申请实施例提供的一种第二图像的示意图;
图13是本申请实施例提供的又一种图像处理设备的显示界面示意图;
图14是本申请实施例提供的再一种图像处理设备的显示界面示意图;
图15是本申请实施例提供的一种图像处理设备的结构示意图;
图16是本申请实施例提供的另一种图像处理设备的结构示意图;
图17是本申请实施例提供的又一种图像处理设备的结构示意图;
图18是本申请实施例提供的再一种图像处理设备的结构示意图;
图19是本申请实施例提供的还一种图像处理设备的结构示意图;
图20是本申请实施例提供的一种图像处理设备的框图。
具体实施方式
为使本申请的目的、技术方案和优点更加清楚,下面将结合附图对本申请实施方式作进一步地详细描述。
为了便于读者对本申请技术方案的理解,首先对本申请实施例涉及的部分名词进行介绍。
1、图像坐标系:图像坐标系是以相机采集到的图像的左上顶点为坐标原点的坐标系。图像坐标系的x轴(横轴)与y轴(纵轴)分别为相机采集到的图像的宽高方向。
2、相机坐标系:相机坐标系是以相机的光心为原点,以主光轴为z轴建立的三维直角坐标系。其中,相机坐标系的x轴与该相机采集的图像对应的图像坐标系的x轴平行。相机坐标系的y轴与该相机采集的图像对应的图像坐标系的y轴平行。
3、世界坐标系:世界坐标系能够描述相机在现实世界中的位置,同样还能够描述相机采集到的图像中的物体在现实世界中的位置。基于相机在世界坐标系下的位姿,能够对该相机对应的相机坐标系与世界坐标系进行相互转换。
4、位姿:相机在世界坐标系下的位姿用于描述相机的位置和姿态。位置指相机在世界坐标系下的坐标,可以用平移矩阵来表示。姿态指相机在世界坐标系下的朝向,可以用旋转矩阵来表示。基于相机在世界坐标系下的位姿可计算得到相机外参。
5、相机参数:相机参数可采用相机矩阵来表示。
在一种实现方式中,相机参数可以基于相机内参和相机外参计算得到。相机内参可表示为内参矩阵K:
Figure PCTCN2022078656-appb-000001
其中,f x和f y表示相机焦距,与像元尺寸有关。c x表示相机的主光轴在该相机采集的图像 对应的图像坐标系的x轴方向的偏移量,即c x为相机的光心在该相机采集的图像对应的图像坐标系中的横坐标,通常为该相机采集的图像的宽的一半。c y表示相机的主光轴在该相机采集的图像对应的图像坐标系的y轴上的偏移量,即c y为相机的光心在该相机采集的图像对应的图像坐标系中的纵坐标,通常为该相机采集的图像的高的一半。
相机外参可表示为外参矩阵T:
T=[R t]     公式(2)
其中,R是旋转矩阵,是一个3行3列的矩阵,用来表示相机的姿态。t是平移矩阵,是一个3行1列的矩阵,用来表示相机的位置。
相机的内参矩阵K和外参矩阵T一起组成相机矩阵W:
W=K×T      公式(3)
在另一种实现方式中,相机参数可以根据多个三维点在世界坐标系下的空间位置以及该多个三维点对应在相机采集到的图像中的成像位置计算得到。第i个三维点在世界坐标系下的空间坐标P i=(x i,y i,z i),第i个三维点在相机采集到的图像中对应的二维坐标s i=(u i,v i),i为正整数。设每个三维点P i=(x i,y i,z i,1)(齐次坐标下)按照以下公式(4)投影到图像中的像素点s i=(u i,v i,1)。
λ is i=WP i      公式(4)
其中,W表示相机矩阵,
Figure PCTCN2022078656-appb-000002
则P i和s i满足的关系见以下公式(5):
Figure PCTCN2022078656-appb-000003
将公式(5)中的i分别取值为0至N,N为多个三维点的数量,N>2,并采用奇异值分解(singular value decomposition,SVD)求解最小二乘问题的方式对公式(5)进行求解,可以得到相机矩阵W。
6、投影变换:采用一个相机的相机参数对世界坐标系下的三维点进行投影变换,可以得到该三维点在该相机采集的图像对应的图像坐标系下对应的像素点。
假设世界坐标系下的三维点的齐次坐标为P w=(X w Y w Z w 1),则通过以下公式可以得到该三维点在图像坐标系下对应的像素点(u,v)。
Figure PCTCN2022078656-appb-000004
在现代会议系统中,随着跨地域的协作越来越多以及成本的要求,对视频会议的需求日益增加。出于对用户所处环境隐私的保护,通常需要将视频会议图像中的真实背景替换为虚拟背景。采用相机拍摄同一场景时,如果相机的位置或姿态不同,那么相机采集到的图像的视角和可视区域也会不同。目前会议室中可用来部署相机的位置通常有多个,在采用相机拍摄会议室中的参会人员时,将相机放置在一个位置上,拍摄到的参会人员可能是正对相机的,而将相机放置在另一个位置上,拍摄到的参会人员可能是侧对相机的。因此,如果不考虑相机的拍摄位置,而直接采用预先配置的虚拟背景来更换相机拍摄的视频会议图像中的背景, 可能会出现人体成像与虚拟背景不匹配的问题,导致合成图像的成像效果较差。
有鉴于此,本申请实施例提供了一种图像背景生成方法,通过预先配置目标场景对应的模拟场景,在目标场景中部署相机之后,图像处理设备采用该相机的相机参数对模拟场景的三维场景模型进行投影变换,可以得到该相机视角下的图像背景,该图像背景与该相机采集的真实图像的视角和可视区域都是一致的。因此,采用该图像背景更换该相机采集的真实图像中的背景后得到的合成图像中,前景与背景始终都是匹配的,合成图像的成像效果较好。
下面从应用场景、方法流程、软件装置、硬件装置等多个角度,对本申请提供的技术方案进行详细介绍。
下面对本申请实施例的应用场景举例说明。
图1是本申请实施例提供的一种图像背景生成方法涉及的应用场景示意图。如图1所示,该应用场景包括多个会议终端。每个会议终端都可以与其它所有会议终端进行通信。图1示出的应用场景以包括两个会议终端(会议终端1-2)为例,实际场景还可以包括三个、四个或更多的会议终端,本申请实施例对应用场景包括的会议终端的数量不做限制。
可选地,会议终端集成有相机,或者,会议终端与相机连接。本申请实施例中,将会议终端中集成的相机采集的视频流以及与会议终端连接的相机采集的视频流统称为会议终端采集的视频流。可选地,参见图1,会议终端1用于向会议终端2发送会议终端1采集的视频流1,会议终端2用于向会议终端1发送会议终端2采集的视频流2,以实现会议终端1侧的参会方与会议终端2侧的参会方之间的视频会议。
会议终端用于显示图像或播放视频流。可选地,会议终端为大屏、电子白板、手机、平板电脑或智能可穿戴设备等具有显示功能的电子设备。可选地,会议终端还具有图像处理功能,即会议终端可用作图像处理设备。或者,会议终端与服务器、服务器集群或云计算中心(图中未示出)等具有图像处理功能的设备连接,该具有图像处理功能的设备用作图像处理设备,会议终端用作图像显示设备。
本申请实施例提供的图像背景生成方法不仅应用于视频会议场景,还可应用于例如视频聊天等其它视频通讯场景,还可应用于具有图像背景更换需求的其它场景,本申请实施例对应用场景的类型不做限定。
下面对本申请实施例的方法流程举例说明。
图2是本申请实施例提供的一种图像背景生成方法的流程示意图。如图2所示,该方法包括以下步骤201至步骤205。
步骤201、图像处理设备获取目标场景对应的模拟场景的描述信息,该描述信息包括三维场景模型。
可选地,图像处理设备为具有图像处理能力的会议终端或与会议终端连接的具有图像处理能力的设备。本申请实施例以图像处理设备为会议终端为例进行说明,例如,图像处理设备为如图1所示的应用场景中的会议终端。
目标场景为真实场景,模拟场景为模拟目标场景构建的虚拟三维场景。可选地,目标场景为会议室,模拟场景为模拟该会议室构建的虚拟会议室。例如,图3是本申请实施例提供的一种模拟场景的示意图。如图3所示,该模拟场景包括墙体、地面和桌子。模拟场景的三 维场景模型包括形成该模拟场景的三维点的颜色以及三维点在世界坐标系下的坐标。
可选地,图像处理设备中预先存储有目标场景与一个或多个模拟场景的对应关系。如果图像处理设备中存储有目标场景与多个模拟场景的对应关系,那么图像处理设备可以显示该多个模拟场景的俯视图以供用户选择,然后获取用户所选择的模拟场景的描述信息。其中,模拟场景的俯视图可以是采用相机俯拍时的相机参数,对该模拟场景的三维场景模型进行投影变换得到的。例如,图4是本申请实施例提供的一种模拟场景的三维场景模型变换为俯视图的示意图。如图4所示,假设相机部署在三维场景模型的正上方的水平中心,且相机的拍摄角度为俯拍,则采用该相机的相机参数对图4中左图示出的三维场景模型进行投影变换,可以得到图4中右图示出的俯视图。
步骤202、图像处理设备获取部署在目标场景中的相机的相机参数。
在一种实现方式中,步骤202的实现过程包括:图像处理设备根据相机在世界坐标系下的位姿,确定该相机的相机外参。图像处理设备根据该相机外参以及获取的该相机的相机内参,确定该相机的相机参数。可选地,图像处理设备采用标定算法标定相机内参,本申请实施例对所采用的标定算法的类型不做限定。
可选地,模拟场景的描述信息包括模拟场景中的一个或多个参考位置点对应的位姿信息。该一个或多个参考位置点是预先设置的可能会放置相机的位置点,本申请实施例对参考位置点的设置位置和设置数量均不做限定。参考位置点的位姿信息包括参考位置点在世界坐标系下的水平位置和/或参考位置点对应的相机角度。其中,相机角度用来表征相机姿态。参考位置点在世界坐标系下的水平位置可以采用世界坐标系下的水平二维坐标(x,y)表示。参考位置点对应的相机角度可以采用(roll,pitch,yaw)的形式表示,roll表示旋转角,pitch表示俯仰角,yaw表示偏转角。例如,图5是本申请实施例提供的一种相机角度的表示示意图。
可选地,图像处理设备确定相机在世界坐标系下的位姿的实现方式包括以下步骤2021和步骤2022。
在步骤2021中,图像处理设备显示模拟场景的俯视图,并在俯视图上显示参考位置点。
可选地,参考位置点的位姿信息包括参考位置点在世界坐标系下的水平位置以及参考位置点对应的相机角度。例如,图6是本申请实施例提供的一种图像处理设备的显示界面示意图。如图6所示,该俯视图上显示有3个参考位置点,记为A、B和C。以O为原点建立平面坐标系,该平面坐标系为世界坐标系在水平面上的坐标系,即x轴和y轴都位于水平面上。参考位置点A在该平面坐标系下的坐标为(2,2),参考位置点A对应的相机角度为(0°,0°,-45°);参考位置点B在该平面坐标系下的坐标为(2,5),参考位置点B对应的相机角度为(0°,0°,45°);参考位置点C在该平面坐标系下的坐标为(10,3),参考位置点C对应的相机角度为(0°,0°,180°)。其中,水平位置的坐标单位为米。由于该平面坐标系是世界坐标系在水平面上的坐标系,因此参考位置点在该平面坐标系下的坐标也即是该参考位置点在世界坐标系下的水平位置。
在步骤2022中,当图像处理设备接收到针对该一个或多个参考位置点中的目标参考位置点的选择指令时,图像处理设备根据该目标参考位置点对应的位姿信息,确定相机在世界坐标系下的位姿。
可选地,参考位置点的位姿信息包括参考位置点在世界坐标系下的水平位置。步骤2022的实现方式包括以下步骤S221至步骤S223。
在步骤S221中,当图像处理设备接收到针对目标参考位置点的选择指令时,图像处理设备显示目标参考位置区域,该目标参考位置区域内包括目标参考位置点以及位于目标参考位置点周围的一个或多个可选位置点。
例如,图7是本申请实施例提供的另一种图像处理设备的显示界面示意图。如图7所示,当图像处理设备接收到针对图6中的参考位置点C的选择指令时,可以显示以参考位置点C为圆心的参考位置区域,该参考位置区域内包括参考位置点C以及多个可选位置点(以方格顶点表示)。
本申请实施例中,当用户在图像处理设备的显示界面上选择一个参考位置点时,图像处理设备可以以被选择的参考位置点为中心,辐射出被选择的参考位置点附近的其它位置点,通过用户界面(user interface,UI)交互的方式,实现用户对确定的相机位置的可调性。
可选地,当图像处理设备接收到针对目标参考位置点的选择指令时,图像处理设备显示目标参考位置点对应的图像背景。目标参考位置点对应的图像背景是指,模拟场景在目标参考位置点对应的相机视角下投影得到的图像。图像处理设备可以将目标参考位置点的位姿信息作为相机的部分位姿信息,并结合相机安装高度来确定相机外参,然后采用相机内参以及相机外参对三维场景模型进行投影变换,得到目标参考位置点对应的图像背景。例如请继续参考图7,图像处理设备的显示界面上还显示有参考位置点C对应的图像背景。
本申请实施例中,当用户在图像处理设备的显示界面上选择一个参考位置点时,图像处理设备可以显示该参考位置点对应的图像背景,便于用户基于参考位置点对应的图像背景,直观地调整选择的相机位置。
在步骤S222中,当图像处理设备接收到针对目标位置参考点的确认指令时,图像处理设备将目标位置参考点在世界坐标系下的水平位置作为相机在世界坐标系下的水平位置。
在步骤S223中,当图像处理设备接收到针对该一个或多个可选位置点中的目标可选位置点的确认指令时,图像处理设备根据目标参考位置点在世界坐标系下的水平位置以及目标可选位置点与目标参考位置点的相对位置,确定目标可选位置点在世界坐标系下的水平位置,并将目标可选位置点在世界坐标系下的水平位置作为相机在世界坐标系下的水平位置。
例如,假设图7示出的参考位置区域内任意相邻两个位置点之间的距离均为0.5米,则位于参考位置点C的右侧、且与参考位置点C相邻的可选位置点在世界坐标系下的水平位置可表示为(10.5,3)。如果图像处理设备接收到针对该可选位置点的确认指令,则确定部署在目标场景中的相机在世界坐标系下的水平位置为(10.5,3)。
可选地,当图像处理设备接收到针对可选位置点的选择指令时,终端还可以以被选择的可选位置点为中心,继续辐射出被选择的可选位置点附近的其它位置点,以此往复,直至用户确定位置点为止。
可选地,选择指令的触发条件与确认指令的触发条件不同。例如,当图像处理设备检测到对某个位置点的单击操作时,图像处理设备确定接收到针对该位置点的选择指令。当图像处理设备检测到对某个位置点的双击操作时,图像处理设备确定接收到针对该位置点的确认指令。又例如,当图像处理设备检测到对某个位置点的单击操作时,图像处理设备确定接收到针对该位置点的选择指令;当图像处理设备再次检测到对该位置点的单击操作时,图像处理设备确定接收到针对该位置点的确认指令。
本申请实施例中,上述步骤S221至步骤S223也可替代实现为:当图像处理设备接收到 针对目标参考位置点的选择指令时,图像处理设备直接将该目标参考位置点在世界坐标系下的水平位置确定为相机在世界坐标系下的水平位置。
在上述步骤S221至步骤S223中,图像处理设备获取了相机在世界坐标系下的水平位置。可选地,图像处理设备可以对相机在世界坐标系下的高度进行初始化。例如,图像处理设备为大屏,相机集成在大屏的顶部,则图像处理设备可以将相机安装高度初始化为大屏的顶部高度。可选地,图像处理设备还可以提示用户输入相机安装高度,并支持手动更改相机安装高度的功能。如果图像处理设备接收到输入的相机安装高度,则图像处理设备将该输入的相机安装高度作为相机在世界坐标系下的高度。
进一步地,图像处理设备可以基于相机在世界坐标系下的水平位置以及相机在世界坐标系下的高度得到该相机对应的平移矩阵。
可选地,参考位置点的位姿信息包括参考位置点对应的相机角度。步骤2022的实现方式包括:当图像处理设备接收到针对目标参考位置点的选择指令时,图像处理设备基于目标参考位置点对应的相机角度计算得到部署在目标场景中的相机对应的旋转矩阵。
在得到相机对应的平移矩阵和旋转矩阵之后,基于上述公式(3)可以计算得到相机参数(即相机矩阵)。
在另一种实现方式中,目标场景中设置有多个标志物,图像处理设备中存储有多个标志物在世界坐标系下的空间位置。步骤202的实现过程包括:图像处理设备获取目标图像,该目标图像由相机对目标场景拍摄得到。图像处理设备根据多个标志物在目标图像中的成像位置以及该多个标志物在世界坐标系下的空间位置,确定相机参数。
可选地,可以预先配置模拟场景中的多个锚点的空间位置。在相机安装完成后,图像处理设备可以提示用户在目标场景中的多个锚点处分别设置标志物。例如,图像处理设备可以显示模拟场景以及模拟场景中的各个锚点,以便用户直观地获取标志物所需设置位置。或者,在相机安装完成后,用户也可以自行在目标场景中设置多个标志物,并向图像处理设备输入该多个标志物的空间位置。
在目标场景中设置完成标志物之后,可以启用相机对目标场景进行拍摄。图像处理设备在获取相机对目标场景拍摄得到的图像之后,可以采用图像识别算法识别出图像中的标志物成像,并将标志物成像的中心作为标志物的成像位置。进一步根据多个标志物在图像中的成像位置以及多个标志物的空间位置,基于上述公式(5)计算得到相机参数(即相机矩阵)。
本实现方式中,图像处理设备根据多个标志物在世界坐标系下的空间位置以及在目标图像中的成像位置自动确定相机参数,而无需用户手动选择相机的部署位置,可提升用户体验。另外,当相机在目标场景中的位姿发生变化时,也无需重新设置标志物或更改锚点的空间位置,图像处理设备可根据相机重新拍摄的图像以及锚点的空间位置自行计算出变更后的相机参数,操作便捷性更好。
步骤203、图像处理设备采用相机参数对三维场景模型进行投影变换,得到该相机对应的图像背景。
本申请实施例中,通过预先配置目标场景对应的模拟场景,在目标场景中部署相机之后,图像处理设备采用该相机的相机参数对模拟场景的三维场景模型进行投影变换,可以得到该相机视角下的图像背景,该图像背景与该相机采集的真实图像的视角和可视区域都是一致的。因此,采用该图像背景更换该相机采集的真实图像中的背景后得到的合成图像中,前景与背 景始终都是匹配的,合成图像的成像效果较好。
可选地,模拟场景的三维场景模型包括第一背景模型和第二背景模型。第二背景模型位于第一背景模型所形成的空间内。例如,第一背景模型包括墙体、地面等。第二背景模型包括桌子、椅子等。步骤203的实现过程包括以下步骤2031至步骤2033。
在步骤2031中,图像处理设备采用相机参数对第一背景模型进行投影变换,得到第一子图像背景。
例如结合参考图3和图6,第一背景模型包括图3示出的模拟场景中的墙体和地面,如果相机在目标场景中的部署位置与图6示出的参考位置点C对应,则图像处理设备采用相机参数对第一背景模型进行投影变换,可以得到如图8所示的第一子图像背景。
在步骤2032中,图像处理设备采用相机参数对第二背景模型进行投影变换,得到第二子图像背景。
例如结合参考图3和图6,第二背景模型包括图3示出的模拟场景中的桌子,如果相机在目标场景中的部署位置与图6示出的参考位置点C对应,则图像处理设备采用相机参数对第二背景模型进行投影变换,可以得到如图9所示的第二子图像背景。
在步骤2033中,图像处理设备将第二子图像背景叠加在第一子图像背景上,得到图像背景。
将第二子图像背景叠加在第一子图像背景上,是指将包含第二子图像背景的图层叠加在包含第一子图像背景的图层上,即第二子图像背景和第一子图像背景在不同图层中,这样可以实现对第二子图像背景的单独调节。例如,图像处理设备将图9示出的第二子图像背景叠加在图8示出的第一子图像背景上,可以得到如图10所示的图像背景。
或者,模拟场景的三维场景模型为一个整体模型,则图像处理设备可以采用相机参数对三维场景模型进行整体投影变换,得到相机对应的图像背景。
可选地,图像处理设备在得到相机对应的图像背景之后,还可以执行以下步骤204和步骤205。
步骤204、图像处理设备获取第一图像中的前景成像,该第一图像由该相机采集得到。
本申请实施例中,第一图像可以是将目标相机固定部署在目标场景中后由目标相机采集得到的任一图像。
可选地,步骤204的实现方式包括:图像处理设备对第一图像进行人体语义分割,得到第一图像中的人体掩膜。然后图像处理设备根据人体掩膜以及第一图像,得到前景成像。例如,图11是本申请实施例提供的一种第一图像中的前景成像的生成过程示意图。
步骤205、图像处理设备对第一图像中的前景成像以及该相机对应的图像背景进行融合,得到第二图像。
第二图像为合成图像。可选地,第一图像、第二图像以及图像背景满足公式:I=αF+(1-α)B。其中,I为表征第二图像的像素矩阵,α为人体掩膜,F为表征第一图像的像素矩阵,B为表征图像背景的像素矩阵。例如,将图11示出的前景成像与图10示出的图像背景进行融合,可以得到如图12所示的第二图像。
当图像处理设备为会议终端时,会议终端得到第二图像之后,进一步可以将第二图像发送给远端会议终端,以实现视频会议通信。
综上所述,在本申请实施例提供的图像背景生成方法中,通过预先配置目标场景对应的 模拟场景,在目标场景中部署相机之后,图像处理设备采用该相机的相机参数对模拟场景的三维场景模型进行投影变换,可以得到该相机视角下的图像背景,该图像背景与相机采集的真实图像的视角和可视区域都是一致的。因此,采用该图像背景更换该相机采集的真实图像中的背景后得到的合成图像中,前景与背景始终都是匹配的,合成图像的成像效果较好。本申请方案不受限于相机的部署位置以及相机的姿态,灵活性较高。可选地,在确定相机位置进而确定相机外参的过程中,当用户在图像处理设备的显示界面上选择一个参考位置点时,图像处理设备可以以被选择的参考位置点为中心,辐射出被选择的参考位置点附近的其它位置点,通过UI交互的方式,实现用户对确定的相机位置的可调性。
可选地,本申请实施例中,图像处理设备在得到相机对应的图像背景之后,还可以通过UI交互的方式对图像背景进行调整。相应地,上述步骤205中,图像处理设备可以对第一图像中的前景成像以及调整后的图像背景进行融合,得到第二图像。
在一种可能实现方式中,图像处理设备可以对图像背景进行整体调整,具体实现过程包括以下步骤S11至步骤S12:
在步骤S11中,图像处理设备显示图像背景以及背景调节控件。
背景调节控件用于调节背景方向、背景位置和/或背景可视区域。其中,背景方向的调节可以通过调整相机外参中的旋转矩阵实现。背景位置的调节可以通过调整相机外参中的平移矩阵实现。背景可视区域的调节可以通过调整相机的焦距实现。调节背景方向和调节背景位置实际上都是调节背景视角。可选地,背景调节控件还可以用于调节背景大小,通过对图像背景进行缩放以实现对图像背景的大小调节。
可选地,图像处理设备可以获取部署在目标场景中的相机采集的图像中的前景成像,并将前景成像叠加在得到的图像背景上,以得到预合成图像。然后图像处理设备显示预合成图像以及针对预合成图像中图像背景的背景调节控件。例如,图13是本申请实施例提供的又一种图像处理设备的显示界面示意图。如图13所示,图像处理设备上显示有预合成图像以及背景编辑界面,该背景编辑界面包括背景方向调节控件、背景位置调节控件和背景可视区域调节控件。其中,背景方向调节控件采用旋转轴x、y、z表示,分别对应俯仰角、偏转角和旋转角。背景位置调节控件采用位置可上下、左右可移动的小方块表示。背景可视区域调节控件采用半径可调的圆表示,增大圆半径表示放大背景,减小圆半径表示缩小背景。
在步骤S12中,响应于对背景调节控件的第一操控指令,图像处理设备根据第一操控指令调节图像背景。
例如,响应于对背景方向调节控件的操控指令,图像处理设备根据该操控指令调整相机外参中的旋转矩阵得到新的相机参数,然后再次执行上述步骤203,从而实现对背景方向的调节。又例如,响应于对背景位置调节控件的操控指令,图像处理设备根据该操控指令调整相机外参中的平移矩阵得到新的相机参数,然后再次执行上述步骤203,从而实现对背景位置的调节。又例如,响应于对背景可视区域调节控件的操控指令,图像处理设备根据该操控指令调整相机的焦距以调整相机内参得到新的相机参数,然后再次执行上述步骤203,从而实现对背景可视区域的调节。
本申请实施例中,图像处理设备显示图像背景以及针对图像背景的背景调节控件,通过UI交互的方式实现对图像背景的整体调整。另外,图像处理设备可以以预合成图像的形式显 示图像背景,使得用户可以参考预合成图像中的前景成像对图像背景进行调整,以使调整后的图像背景能够更好地匹配相机视角,进而使调整后的图像背景能够更好地匹配相机采集的真实图像中的前景。
在另一种可能实现方式中,如果上述步骤203中图像处理设备获取的图像背景是将第二子图像背景叠加在第一子图像背景上得到的,即图像背景中的第一子图像背景和第二子图像背景在不同图层上,那么图像处理设备可以对图像背景进行局部调整,具体实现过程包括以下步骤S21至步骤S22:
在步骤S21中,图像处理设备显示图像背景以及针对图像背景中的第二子图像背景的调节控件。
针对第二子图像背景的调节控件用于调节第二子图像背景的位置和/或大小。
可选地,图像处理设备可以获取部署在目标场景中的相机采集的图像中的前景成像,并将前景成像叠加在得到的图像背景上,以得到预合成图像。然后图像处理设备显示预合成图像以及针对预合成图像中第二子图像背景的调节控件。例如,第二子图像背景为桌子成像,图14是本申请实施例提供的再一种图像处理设备的显示界面示意图。如图14所示,图像处理设备上显示有预合成图像以及针对桌子成像的操作轴,通过移动该操作轴可以移动桌子成像的位置,通过缩放该操作轴可以对桌子成像进行缩放。
在步骤S22中,响应于对调节控件的第二操控指令,图像处理设备根据第二操控指令调节第二子图像背景。
本申请实施例中,图像处理设备显示图像背景以及针对图像背景中的子图像背景的调节控件,通过UI交互的方式实现对图像背景的局部调整,提高了对图像背景的调整灵活性。另外,图像处理设备可以以预合成图像的形式显示图像背景,使得用户可以参考预合成图像中的前景成像对图像背景进行局部调整,以使调整后的图像背景能够更好地匹配相机采集的真实图像中的前景。
可选地,图像处理设备得到图像背景之后,也可以在同一界面上同时显示图像背景、背景调节控件以及针对图像背景中的第二子图像背景的调节控件,本申请实施例对此不做限定。
可选地,图像处理设备在对图像背景进行调整之后,可以存储调整后的图像背景对应的相机参数和/或调整后的图像背景,以便图像处理设备后续可以直接使用存储的相机参数或图像背景,降低用户再次调整图像背景的概率,从而提高用户体验。
可选地,当图像处理设备中存储有相机参数以及一个或多个图像背景时,图像处理设备获取图像背景的实现过程可以包括以下步骤S31至步骤S34。
在步骤S31中,图像处理设备显示第一选项、第二选项和第三选项。
其中,第一选项用于指示初始化相机参数,第二选项用于指示使用图像处理设备存储的相机参数,第三选项用于指示使用图像处理设备存储的图像背景。
在步骤S32中,如果图像处理设备接收到对第一选项的选择指令,则图像处理设备执行上述步骤201至步骤203,以得到图像背景。
在步骤S33中,如果图像处理设备接收到对第二选项的选择指令,则图像处理设备获取存储的相机参数并执行上述步骤201,然后采用存储的相机参数对模拟场景的三维场景模型进行投影变换,得到图像背景。
在步骤S34中,如果图像处理设备接收到对第三选项的选择指令,则图像处理设备获取 并显示存储的图像背景,然后根据用户的选择确定最终使用的图像背景。
本申请实施例提供的上述图像背景生成方法的步骤先后顺序可以进行适当调整,步骤也可以根据情况进行相应增减。任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到变化的方法,都应涵盖在本申请的保护范围之内。例如,本申请上述实施例均以图像处理设备具有显示功能为例进行说明,实际场景中,上述方法也可以由图像处理设备和图像显示设备配合完成,例如图像处理设备用于执行图像处理动作,图像显示设备用于执行图像显示动作。
下面对本申请实施例的软件装置举例说明。
图15是本申请实施例提供的一种图像处理设备的结构示意图。如图15所示,图像处理设备1500包括:
第一获取模块1501,用于获取目标场景对应的模拟场景的描述信息,该描述信息包括三维场景模型。
第二获取模块1502,用于获取部署在目标场景中的相机的相机参数。
投影变换模块1503,用于采用相机参数对三维场景模型进行投影变换,得到相机对应的图像背景。
可选地,第二获取模块1502,用于根据相机在世界坐标系下的位姿,确定该相机的相机外参。并根据该相机外参以及获取的该相机的相机内参,确定相机参数。
可选地,描述信息还包括模拟场景中的一个或多个参考位置点对应的位姿信息,该位姿信息包括参考位置点在世界坐标系下的水平位置和/或参考位置点对应的相机角度。如图16所示,图像处理设备还包括:显示模块1504,用于显示模拟场景的俯视图,并在俯视图上显示参考位置点。确定模块1505,用于当图像处理设备接收到针对一个或多个参考位置点中的目标参考位置点的选择指令时,根据目标参考位置点对应的位姿信息,确定相机在世界坐标系下的位姿。
可选地,位姿信息包括参考位置点在世界坐标系下的水平位置。显示模块1504,还用于当图像处理设备接收到针对目标参考位置点的选择指令时,显示目标参考位置区域,目标参考位置区域内包括目标参考位置点以及位于目标参考位置点周围的一个或多个可选位置点。确定模块1505,用于当图像处理设备接收到针对一个或多个可选位置点中的目标可选位置点的确认指令时,根据目标参考位置点在世界坐标系下的水平位置以及目标可选位置点与目标参考位置点的相对位置,确定目标可选位置点在世界坐标系下的水平位置,并将目标可选位置点在世界坐标系下的水平位置作为相机在世界坐标系下的水平位置。
可选地,目标场景中设置有多个标志物,图像处理设备中存储有多个标志物在世界坐标系下的空间位置。第二获取模块1502,用于:获取目标图像,该目标图像由相机对目标场景拍摄得到。根据多个标志物在目标图像中的成像位置以及多个标志物在世界坐标系下的空间位置,确定相机参数。
可选地,如图17所示,图像处理设备1500还包括:显示模块1504,用于显示图像背景以及背景调节控件,背景调节控件用于调节背景方向、背景位置和/或背景可视区域。第一背景调节模块1506,用于响应于对背景调节控件的第一操控指令,根据第一操控指令调节图像背景。
可选地,三维场景模型包括第一背景模型和第二背景模型,第二背景模型位于第一背景模型所形成的空间内。投影变换模块1503,用于:采用相机参数对第一背景模型进行投影变换,得到第一子图像背景;采用相机参数对第二背景模型进行投影变换,得到第二子图像背景;将第二子图像背景叠加在第一子图像背景上,得到图像背景。如图18所示,图像处理设备1500还包括:显示模块1504,用于显示图像背景以及针对第二子图像背景的调节控件,调节控件用于调节第二子图像背景的位置和/或大小。第二背景调节模块1507,用于响应于对调节控件的第二操控指令,根据第二操控指令调节第二子图像背景。
可选地,如图19所示,图像处理设备1500还包括:第三获取模块1508,用于获取第一图像中的前景成像,第一图像由相机采集得到。图像融合模块1509,用于对前景成像以及图像背景进行融合,得到第二图像。
关于上述实施例中的装置,其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述,此处将不做详细阐述说明。
下面对本申请实施例的硬件装置举例说明。
图20是本申请实施例提供的一种图像处理设备的框图。该图像处理设备可以是会议终端或与会议终端连接的具有图像处理功能的设备。会议终端可以是大屏、电子白板、手机、平板电脑或智能可穿戴设备等具有显示功能的电子设备,与会议终端连接的具有图像处理功能的设备可以是一台服务器,或者是由若干台服务器组成的服务器集群,或者是一个云计算中心。如图20所示,该图像处理设备2000包括:处理器2001和存储器2002。
存储器2002,用于存储计算机程序,所述计算机程序包括程序指令;
处理器2001,用于调用所述计算机程序,实现如图2所示的方法。
可选地,该图像处理设备2000还包括通信总线2003和通信接口2004。
其中,处理器2001包括一个或者一个以上处理核心,处理器2001通过运行计算机程序,执行各种功能应用以及数据处理。
存储器2002可用于存储计算机程序。可选地,存储器可存储操作系统和至少一个功能所需的应用程序单元。操作系统可以是实时操作系统(Real Time eXecutive,RTX)、LINUX、UNIX、WINDOWS或OS X之类的操作系统。
通信接口2004可以为多个,通信接口2004用于与其它存储设备或网络设备进行通信。例如在本申请实施例中,会议终端的通信接口可以用于向另一个会议终端发送图像或视频流。网络设备可以是交换机或路由器等。
存储器2002与通信接口2004分别通过通信总线2003与处理器2001连接。
本申请实施例还提供了一种计算机可读存储介质,所述计算机可读存储介质上存储有指令,当所述指令被处理器执行时,实现如图2所示的图像背景生成方法。
本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。
在本申请实施例中,术语“第一”、“第二”和“第三”仅用于描述目的,而不能理解为指示或暗示相对重要性。
本申请中术语“和/或”,仅仅是一种描述关联对象的关联关系,表示可以存在三种关系,例如,A和/或B,可以表示:单独存在A,同时存在A和B,单独存在B这三种情况。另外,本文中字符“/”,一般表示前后关联对象是一种“或”的关系。
以上所述仅为本申请的可选实施例,并不用以限制本申请,凡在本申请的构思和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。

Claims (18)

  1. 一种图像背景生成方法,其特征在于,所述方法包括:
    图像处理设备获取目标场景对应的模拟场景的描述信息,所述描述信息包括三维场景模型;
    所述图像处理设备获取部署在所述目标场景中的相机的相机参数;
    所述图像处理设备采用所述相机参数对所述三维场景模型进行投影变换,得到所述相机对应的图像背景。
  2. 根据权利要求1所述的方法,其特征在于,所述图像处理设备获取部署在所述目标场景中的相机的相机参数,包括:
    所述图像处理设备根据所述相机在世界坐标系下的位姿,确定所述相机的相机外参;
    所述图像处理设备根据所述相机外参以及获取的所述相机的相机内参,确定所述相机参数。
  3. 根据权利要求2所述的方法,其特征在于,所述描述信息还包括所述模拟场景中的一个或多个参考位置点对应的位姿信息,所述位姿信息包括所述参考位置点在所述世界坐标系下的水平位置和/或所述参考位置点对应的相机角度,所述方法还包括:
    所述图像处理设备显示所述模拟场景的俯视图,并在所述俯视图上显示所述参考位置点;
    当所述图像处理设备接收到针对所述一个或多个参考位置点中的目标参考位置点的选择指令时,所述图像处理设备根据所述目标参考位置点对应的位姿信息,确定所述相机在所述世界坐标系下的位姿。
  4. 根据权利要求3所述的方法,其特征在于,所述位姿信息包括所述参考位置点在所述世界坐标系下的水平位置,所述当所述图像处理设备接收到针对所述一个或多个参考位置点中的目标参考位置点的选择指令时,所述图像处理设备根据所述目标参考位置点对应的位姿信息,确定所述相机在所述世界坐标系下的位姿,包括:
    当所述图像处理设备接收到针对所述目标参考位置点的选择指令时,所述图像处理设备显示目标参考位置区域,所述目标参考位置区域内包括所述目标参考位置点以及位于所述目标参考位置点周围的一个或多个可选位置点;
    当所述图像处理设备接收到针对所述一个或多个可选位置点中的目标可选位置点的确认指令时,所述图像处理设备根据所述目标参考位置点在所述世界坐标系下的水平位置以及所述目标可选位置点与所述目标参考位置点的相对位置,确定所述目标可选位置点在所述世界坐标系下的水平位置,并将所述目标可选位置点在所述世界坐标系下的水平位置作为所述相机在所述世界坐标系下的水平位置。
  5. 根据权利要求1所述的方法,其特征在于,所述目标场景中设置有多个标志物,所述图像处理设备中存储有所述多个标志物在世界坐标系下的空间位置,所述图像处理设备获取部 署在所述目标场景中的相机的相机参数,包括:
    所述图像处理设备获取目标图像,所述目标图像由所述相机对所述目标场景拍摄得到;
    所述图像处理设备根据所述多个标志物在所述目标图像中的成像位置以及所述多个标志物在所述世界坐标系下的空间位置,确定所述相机参数。
  6. 根据权利要求1至5任一所述的方法,其特征在于,所述方法还包括:
    所述图像处理设备显示所述图像背景以及背景调节控件,所述背景调节控件用于调节背景方向、背景位置和/或背景可视区域;
    响应于对所述背景调节控件的第一操控指令,所述图像处理设备根据所述第一操控指令调节所述图像背景。
  7. 根据权利要求1至6任一所述的方法,其特征在于,所述三维场景模型包括第一背景模型和第二背景模型,所述第二背景模型位于所述第一背景模型所形成的空间内,所述图像处理设备采用所述相机参数对所述三维场景模型进行投影变换,得到所述相机对应的图像背景,包括:
    所述图像处理设备采用所述相机参数对所述第一背景模型进行投影变换,得到第一子图像背景;
    所述图像处理设备采用所述相机参数对所述第二背景模型进行投影变换,得到第二子图像背景;
    所述图像处理设备将所述第二子图像背景叠加在所述第一子图像背景上,得到所述图像背景;
    所述方法还包括:
    所述图像处理设备显示所述图像背景以及针对所述第二子图像背景的调节控件,所述调节控件用于调节所述第二子图像背景的位置和/或大小;
    响应于对所述调节控件的第二操控指令,所述图像处理设备根据所述第二操控指令调节所述第二子图像背景。
  8. 根据权利要求1至7任一所述的方法,其特征在于,所述方法还包括:
    所述图像处理设备获取第一图像中的前景成像,所述第一图像由所述相机采集得到;
    所述图像处理设备对所述前景成像以及所述图像背景进行融合,得到第二图像。
  9. 一种图像处理设备,其特征在于,包括:
    第一获取模块,用于获取目标场景对应的模拟场景的描述信息,所述描述信息包括三维场景模型;
    第二获取模块,用于获取部署在所述目标场景中的相机的相机参数;
    投影变换模块,用于采用所述相机参数对所述三维场景模型进行投影变换,得到所述相机对应的图像背景。
  10. 根据权利要求9所述的图像处理设备,其特征在于,所述第二获取模块,用于:
    根据所述相机在世界坐标系下的位姿,确定所述相机的相机外参;
    根据所述相机外参以及获取的所述相机的相机内参,确定所述相机参数。
  11. 根据权利要求10所述的图像处理设备,其特征在于,所述描述信息还包括所述模拟场景中的一个或多个参考位置点对应的位姿信息,所述位姿信息包括所述参考位置点在所述世界坐标系下的水平位置和/或所述参考位置点对应的相机角度,所述图像处理设备还包括:
    显示模块,用于显示所述模拟场景的俯视图,并在所述俯视图上显示所述参考位置点;
    确定模块,用于当所述图像处理设备接收到针对所述一个或多个参考位置点中的目标参考位置点的选择指令时,根据所述目标参考位置点对应的位姿信息,确定所述相机在所述世界坐标系下的位姿。
  12. 根据权利要求11所述的图像处理设备,其特征在于,所述位姿信息包括所述参考位置点在所述世界坐标系下的水平位置;
    所述显示模块,还用于当所述图像处理设备接收到针对所述目标参考位置点的选择指令时,显示目标参考位置区域,所述目标参考位置区域内包括所述目标参考位置点以及位于所述目标参考位置点周围的一个或多个可选位置点;
    所述确定模块,用于当所述图像处理设备接收到针对所述一个或多个可选位置点中的目标可选位置点的确认指令时,根据所述目标参考位置点在所述世界坐标系下的水平位置以及所述目标可选位置点与所述目标参考位置点的相对位置,确定所述目标可选位置点在所述世界坐标系下的水平位置,并将所述目标可选位置点在所述世界坐标系下的水平位置作为所述相机在所述世界坐标系下的水平位置。
  13. 根据权利要求9所述的图像处理设备,其特征在于,所述目标场景中设置有多个标志物,所述图像处理设备中存储有所述多个标志物在世界坐标系下的空间位置,所述第二获取模块,用于:
    获取目标图像,所述目标图像由所述相机对所述目标场景拍摄得到;
    根据所述多个标志物在所述目标图像中的成像位置以及所述多个标志物在所述世界坐标系下的空间位置,确定所述相机参数。
  14. 根据权利要求9至13任一所述的图像处理设备,其特征在于,所述图像处理设备还包括:
    显示模块,用于显示所述图像背景以及背景调节控件,所述背景调节控件用于调节背景方向、背景位置和/或背景可视区域;
    第一背景调节模块,用于响应于对所述背景调节控件的第一操控指令,根据所述第一操控指令调节所述图像背景。
  15. 根据权利要求9至14任一所述的图像处理设备,其特征在于,所述三维场景模型包括第一背景模型和第二背景模型,所述第二背景模型位于所述第一背景模型所形成的空间内,所述投影变换模块,用于:
    采用所述相机参数对所述第一背景模型进行投影变换,得到第一子图像背景;
    采用所述相机参数对所述第二背景模型进行投影变换,得到第二子图像背景;
    将所述第二子图像背景叠加在所述第一子图像背景上,得到所述图像背景;
    所述图像处理设备还包括:
    显示模块,用于显示所述图像背景以及针对所述第二子图像背景的调节控件,所述调节控件用于调节所述第二子图像背景的位置和/或大小;
    第二背景调节模块,用于响应于对所述调节控件的第二操控指令,根据所述第二操控指令调节所述第二子图像背景。
  16. 根据权利要求9至15任一所述的图像处理设备,其特征在于,所述图像处理设备还包括:
    第三获取模块,用于获取第一图像中的前景成像,所述第一图像由所述相机采集得到;
    图像融合模块,用于对所述前景成像以及所述图像背景进行融合,得到第二图像。
  17. 一种图像处理设备,其特征在于,包括:处理器和存储器;
    所述存储器,用于存储计算机程序,所述计算机程序包括程序指令;
    所述处理器,用于调用所述计算机程序,实现如权利要求1至8任一所述的图像背景生成方法。
  18. 一种计算机可读存储介质,其特征在于,所述计算机可读存储介质上存储有指令,当所述指令被处理器执行时,实现如权利要求1至8任一所述的图像背景生成方法。
PCT/CN2022/078656 2021-08-13 2022-03-01 图像背景生成方法及装置、计算机可读存储介质 WO2023015868A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110932990.2 2021-08-13
CN202110932990.2A CN115705660A (zh) 2021-08-13 2021-08-13 图像背景生成方法及装置、计算机可读存储介质

Publications (1)

Publication Number Publication Date
WO2023015868A1 true WO2023015868A1 (zh) 2023-02-16

Family

ID=85180241

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/078656 WO2023015868A1 (zh) 2021-08-13 2022-03-01 图像背景生成方法及装置、计算机可读存储介质

Country Status (2)

Country Link
CN (1) CN115705660A (zh)
WO (1) WO2023015868A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116311225A (zh) * 2023-05-18 2023-06-23 四川新迎顺信息技术股份有限公司 一种基于图像识别的大数据采集方法、数据管理方法及系统

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668126A (zh) * 2008-09-02 2010-03-10 新奥特(北京)视频技术有限公司 一种用于虚拟演播室系统的实现无限蓝箱功能的方法
CN102263772A (zh) * 2010-05-28 2011-11-30 经典时空科技(北京)有限公司 基于三维技术的虚拟会议系统
US20120121137A1 (en) * 2010-11-12 2012-05-17 Fujitsu Limited Image processing apparatus
CN104349111A (zh) * 2013-07-24 2015-02-11 华为技术有限公司 视频会议会场创建方法及系统
KR20160003614A (ko) * 2015-12-22 2016-01-11 우덕명 3차원 리얼타임 가상입체 스튜디오 장치에서의 가상입체 스튜디오 영상 생성 방법
CN107592490A (zh) * 2017-09-11 2018-01-16 广东欧珀移动通信有限公司 视频背景替换方法、装置和移动终端
CN109859100A (zh) * 2019-01-30 2019-06-07 深圳安泰创新科技股份有限公司 虚拟背景的显示方法、电子设备和计算机可读存储介质
JP2020067815A (ja) * 2018-10-24 2020-04-30 キヤノン株式会社 画像処理装置、画像処理方法およびプログラム
CN113973190A (zh) * 2021-10-28 2022-01-25 联想(北京)有限公司 视频虚拟背景图像处理方法、装置及计算机设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101668126A (zh) * 2008-09-02 2010-03-10 新奥特(北京)视频技术有限公司 一种用于虚拟演播室系统的实现无限蓝箱功能的方法
CN102263772A (zh) * 2010-05-28 2011-11-30 经典时空科技(北京)有限公司 基于三维技术的虚拟会议系统
US20120121137A1 (en) * 2010-11-12 2012-05-17 Fujitsu Limited Image processing apparatus
CN104349111A (zh) * 2013-07-24 2015-02-11 华为技术有限公司 视频会议会场创建方法及系统
KR20160003614A (ko) * 2015-12-22 2016-01-11 우덕명 3차원 리얼타임 가상입체 스튜디오 장치에서의 가상입체 스튜디오 영상 생성 방법
CN107592490A (zh) * 2017-09-11 2018-01-16 广东欧珀移动通信有限公司 视频背景替换方法、装置和移动终端
JP2020067815A (ja) * 2018-10-24 2020-04-30 キヤノン株式会社 画像処理装置、画像処理方法およびプログラム
CN109859100A (zh) * 2019-01-30 2019-06-07 深圳安泰创新科技股份有限公司 虚拟背景的显示方法、电子设备和计算机可读存储介质
CN113973190A (zh) * 2021-10-28 2022-01-25 联想(北京)有限公司 视频虚拟背景图像处理方法、装置及计算机设备

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116311225A (zh) * 2023-05-18 2023-06-23 四川新迎顺信息技术股份有限公司 一种基于图像识别的大数据采集方法、数据管理方法及系统
CN116311225B (zh) * 2023-05-18 2023-07-21 四川新迎顺信息技术股份有限公司 一种基于图像识别的大数据采集方法、数据管理方法及系统

Also Published As

Publication number Publication date
CN115705660A (zh) 2023-02-17

Similar Documents

Publication Publication Date Title
US11196984B2 (en) System and method for generating videos
US11019259B2 (en) Real-time generation method for 360-degree VR panoramic graphic image and video
US10650590B1 (en) Method and system for fully immersive virtual reality
US11627251B2 (en) Image processing apparatus and control method thereof, computer-readable storage medium
CN110691175B (zh) 演播室中模拟摄像机运动跟踪的视频处理方法及装置
JPWO2017094543A1 (ja) 情報処理装置、情報処理システム、情報処理装置の制御方法、及び、パラメーターの設定方法
CN107274491A (zh) 一种三维场景的空间操控虚拟实现方法
Tang et al. A system for real-time panorama generation and display in tele-immersive applications
JP7378243B2 (ja) 画像生成装置、画像表示装置および画像処理方法
CN108961423B (zh) 虚拟信息处理方法、装置、设备及存储介质
EP4054186A1 (en) Information processing apparatus, information processing method, and program
WO2023015868A1 (zh) 图像背景生成方法及装置、计算机可读存储介质
CN113347373A (zh) 通过ar空间定位实时制作特效视频的图像处理方法
JP2017156880A (ja) 画像処理装置および画像処理方法
CN110060349B (zh) 一种扩展增强现实头戴式显示设备视场角的方法
JP2006318015A (ja) 画像処理装置および画像処理方法、画像表示システム、並びに、プログラム
JP2019101563A (ja) 情報処理装置、情報処理システム、情報処理方法及びプログラム
CN115423916A (zh) 基于xr技术的沉浸式互动直播构建方法、系统及介质
WO2023005200A1 (zh) 图像生成方法、装置及系统、计算机可读存储介质
KR20200025022A (ko) 가상 현실 체험 서비스 제공 방법, 장치 및 사용자 단말
CN115914501A (zh) 图像背景确定方法及装置、计算机可读存储介质
JP2019062302A (ja) 画像処理装置、画像表示装置及び画像処理プログラム
CN115700770A (zh) 图像生成方法、装置及系统、计算机可读存储介质
KR20120048343A (ko) 파노라마 영상 제공 방법 및 장치
CN113674433A (zh) 混合现实显示方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22854873

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE