WO2022037215A1 - 摄像头、显示设备和摄像头控制方法 - Google Patents

摄像头、显示设备和摄像头控制方法 Download PDF

Info

Publication number
WO2022037215A1
WO2022037215A1 PCT/CN2021/100429 CN2021100429W WO2022037215A1 WO 2022037215 A1 WO2022037215 A1 WO 2022037215A1 CN 2021100429 W CN2021100429 W CN 2021100429W WO 2022037215 A1 WO2022037215 A1 WO 2022037215A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
data
image
special effect
mirror
Prior art date
Application number
PCT/CN2021/100429
Other languages
English (en)
French (fr)
Inventor
姜俊厚
刘晋
于江
吴汉勇
李保成
祖荫鹏
Original Assignee
海信视像科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 海信视像科技股份有限公司 filed Critical 海信视像科技股份有限公司
Publication of WO2022037215A1 publication Critical patent/WO2022037215A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules

Definitions

  • the present application relates to the technical field of cameras, and in particular, to a camera, a display device and a camera control method.
  • the display device can provide the user with playback pictures such as audio, video, and pictures.
  • display devices can not only provide users with the content of live TV programs received through data broadcasting, but also provide users with various application and service content such as online video and online games.
  • the related technology configures a camera on the display device, obtains the local image data collected by the camera through the controller of the display device, processes and displays it, and realizes functions such as video chat, photography, and video recording on the display device. .
  • This application provides some cameras, display devices and camera control methods.
  • the present application provides some display devices, including:
  • the PTZ camera is used to collect image data, and identify the moving target in the image and the position information of the movement in the image according to the image data;
  • the controller is configured to control the camera to rotate to a target angle when the position of the moving object in the image deviates from the preset position, where the target angle is an angle at which the position of the moving object in the image is the preset position.
  • the present application provides some camera control methods, which are applied to display devices, and the methods include:
  • the camera is controlled to rotate to a target angle, where the target angle is an angle at which the position of the moving object in the image is the preset position.
  • this application provides some PTZ cameras, including:
  • an image acquisition module for acquiring image data
  • a first processing module configured to process the image data to obtain preview data for display on the display of the display device
  • the second processing module is used to identify the moving target in the image and the position of the motion in the image according to the image data
  • the pan-tilt control module is configured to control the camera to rotate to a target angle when the position of the moving target in the image deviates from the preset position, and the target angle is such that the position of the moving target in the image is the preset position angle.
  • the present application further provides some display devices, including: a display, a controller, and the pan-tilt camera described in the first aspect, where the controller is connected to the pan-tilt camera.
  • the present application further provides some camera control methods, the methods include:
  • the camera is controlled to rotate to a target angle, where the target angle is an angle at which the position of the moving object in the image is the preset position.
  • the present application provides some data processing methods, the methods comprising:
  • the mirror camera picture and the forward special effect picture superimposed on the mirror camera picture are displayed.
  • the present application provides some data processing methods, the methods include:
  • the forward camera picture and the forward special effect picture superimposed on the forward camera picture are displayed.
  • the present application provides some data processing methods, the methods comprising:
  • the forward camera picture and the forward special effect picture superimposed on the forward camera picture are displayed.
  • the present application provides some data processing methods, the methods comprising:
  • mirrored camera data and the special effect data a mirrored camera picture and a forward special effect picture superimposed on the mirrored camera picture are displayed.
  • FIG. 1 exemplarily shows a schematic diagram of an operation scene between a display device and a control apparatus according to some embodiments
  • FIG. 2 exemplarily shows a hardware configuration block diagram of a display device 200 according to some embodiments
  • FIG. 3 exemplarily shows a hardware configuration block diagram of the control device 100 according to some embodiments
  • FIG. 4 is a schematic diagram of the arrangement structure of a display device and a camera in an embodiment of the present application
  • 5a to 5f are schematic diagrams of the angles of the cameras exemplarily shown in the application;
  • FIG. 6 is a schematic diagram of a possible camera configuration exemplarily shown in the application.
  • FIG. 7 is a schematic diagram of pixel positions in an embodiment of the present application.
  • FIG. 8 is a schematic diagram of key points in the embodiment of the present application.
  • FIG. 9 is a schematic diagram of the geometric relationship of the process of calculating the rotation angle in the embodiment of the present application.
  • FIG. 10 is a schematic diagram of the process of adjusting the rotation angle in the embodiment of the application.
  • FIG. 11 is a schematic diagram of a possible camera configuration exemplarily shown in the application.
  • FIG. 12a is a schematic diagram of an implementation manner of capturing a photo or video with special effect information as exemplarily shown in the application;
  • FIG. 12b is a schematic diagram of another implementation manner of capturing photos or videos with special effect information exemplarily shown in the application;
  • FIG. 13 exemplarily shows the display effect before the text special effect is mirrored and the display effect after being mirrored
  • FIG. 14 exemplarily shows a flowchart of the data processing method in this embodiment of the present application.
  • module refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware or/and software code capable of performing the functions associated with that element.
  • FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to an embodiment. As shown in FIG. 1 , the user can operate the display device 200 through the smart terminal 300 or the control device 100 .
  • the control apparatus 100 may be a remote controller, and the communication between the remote controller and the display device includes infrared protocol communication or Bluetooth protocol communication, and other short-distance communication methods, and the display device 200 is controlled wirelessly or wiredly.
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like.
  • a smart terminal 300 eg, a mobile terminal, a tablet computer, a computer, a notebook computer, etc.
  • the display device 200 is controlled using an application running on the smart device.
  • the display device 200 may also be controlled in a manner other than the control apparatus 100 and the smart device 300.
  • the module for acquiring voice commands configured inside the display device 200 may directly receive the user's voice command for control.
  • the user's voice command control can also be received through a voice control device provided outside the display device 200 device.
  • the display device 200 is also in data communication with the server 400 .
  • the display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks.
  • the server 400 may provide various contents and interactions to the display device 200 .
  • FIG. 2 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply.
  • the control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .
  • FIG. 3 is a block diagram showing a hardware configuration of the display apparatus 200 according to an exemplary embodiment.
  • Display apparatus 200 includes at least some of tuner 210, communicator 220, detector 230, external device interface 240, controller 250, display 275, audio output interface 285, memory, power supply, and user interface.
  • the display 275 includes a display screen component for presenting pictures, and a driving component for driving image display, for receiving image signals output from the controller, components for displaying video content, image content, and menu manipulation interfaces, and user manipulation UI interfaces .
  • the display 275 can be a liquid crystal display, an OLED display, and a projection display, as well as some projection devices and projection screens.
  • the communicator 220 is a component for communicating with external devices or servers according to various communication protocol types.
  • the communicator may include a Wifi module, a Bluetooth module, a wired Ethernet module and other network communication protocol chips or near field communication protocol chips, and at least some of the infrared receivers.
  • the display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the server 400 through the communicator 220 .
  • the user interface can be used to receive control signals from the control device 100 (eg, an infrared remote control, etc.).
  • control device 100 eg, an infrared remote control, etc.
  • the detector 230 is used to collect external environment or external interaction signals.
  • the detector 230 includes a light receiver, a sensor for collecting ambient light intensity; alternatively, the detector 230 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or , the detector 230 includes a sound collector, such as a microphone, for receiving external sound.
  • the external device interface 240 may include but is not limited to the following: any one of a high-definition multimedia interface interface (HDMI), an analog or data high-definition component input interface (component), a composite video input interface (CVBS), a USB input interface (USB), an RGB port, etc. or multiple interfaces. It may also be a composite input/output interface formed by a plurality of the above-mentioned interfaces.
  • HDMI high-definition multimedia interface interface
  • component analog or data high-definition component input interface
  • CVBS composite video input interface
  • USB USB input interface
  • RGB port etc.
  • It may also be a composite input/output interface formed by a plurality of the above-mentioned interfaces.
  • the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.
  • the controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory.
  • the controller 250 controls the overall operation of the display apparatus 200 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 275, the controller 250 may perform an operation related to the object selected by the user command.
  • Objects can be any of the optional objects, such as hyperlinks, icons, or other actionable controls.
  • the operations related to the selected object include: displaying operations connected to hyperlinked pages, documents, images, etc., or executing operations of programs corresponding to the icons.
  • the user may input user commands on a graphical user interface (GUI) displayed on the display 275, and the user input interface receives the user input commands through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
  • GUI Graphical User Interface
  • a system may include a kernel (Kernel), a command parser (shell), a file system, and applications.
  • the kernel, shell, and file system make up the basic operating system structures that allow users to manage files, run programs, and use the system.
  • the kernel starts, activates the kernel space, abstracts hardware, initializes hardware parameters, etc., runs and maintains virtual memory, scheduler, signals and inter-process communication (IPC).
  • IPC inter-process communication
  • the shell and user applications are loaded. An application is compiled into machine code after startup, forming a process.
  • the camera as a detector may be built-in or externally connected to the display device, and after the operation is started, the camera may detect image data.
  • the camera can be connected with the controller through the interface part, so as to send the detected image data to the controller for processing.
  • the angle at which the camera is located is related to the field of view of the camera.
  • the gimbal is a supporting device for installing and fixing the camera. It is divided into fixed gimbal and electric gimbal. Among them, the electric gimbal is suitable for large-scale scanning and shooting, and it can expand the field of view of the camera.
  • the electric gimbal can be a horizontal rotating gimbal that can only rotate left and right, or an omnidirectional gimbal that can rotate left and right as well as up and down.
  • two motors are installed in the omnidirectional pan/tilt head, which are used to drive the pan/tilt head to rotate in the horizontal direction and the vertical direction respectively, so as to change the angle of the camera.
  • the lens assembly may be based on the detection principle of CCD (Charge Coupled Device, charge coupled device) or CMOS (Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor), so as to generate image data of electrical signals according to user images.
  • CCD Charge Coupled Device, charge coupled device
  • CMOS Complementary Metal Oxide Semiconductor, complementary metal oxide semiconductor
  • the display device controller is used to control the electric PTZ, so that the camera can shoot at multiple angles.
  • the camera in the present application can realize all functions related to capturing images, and when the angle rotation is not involved, it is not distinguished from the camera without angle adjustment. When the angle of capturing images can be adjusted, it is a PTZ camera unless otherwise specified.
  • any camera that can realize the functions described in this document is within the scope of the present application, and the structure and components of the camera are not particularly limited.
  • the limit angle that the camera can rotate in the horizontal and/or vertical direction can be designed according to the needs.
  • the rotatable angle of the camera in the horizontal direction may range from 0° to 120°, where 0° and 120° are the corresponding limits of the two rotation directions (leftward and rightward) in the horizontal direction, respectively.
  • Angle; the rotatable angle of the camera in the vertical direction can be from 0° to 180°, where 0° and 180° are respectively the limit angles corresponding to the two rotation directions (up and down) in the vertical direction.
  • Fig. 5a to Fig. 5f are schematic diagrams of the angle of the camera exemplarily shown in the application, wherein Fig. 5a exemplarily shows the state when the tilt angle of the camera in the vertical direction is 0°, Fig. 5b exemplarily shows The state when the tilt angle of the camera in the vertical direction is 90°, Fig. 5c exemplarily shows the state when the tilt angle of the camera in the vertical direction is 105°, and Fig. 5d exemplarily shows the state of the camera in the horizontal direction. The state when the horizontal angle is 0°, Fig. 5e exemplarily shows the state when the horizontal angle of the camera in the horizontal direction is 60°, Fig. 5f exemplarily shows the state when the horizontal angle of the camera in the horizontal direction is 120° status.
  • Embodiments of the present application provide some display devices, including the display in the embodiments shown in FIGS. 1-7 , a PTZ camera, and a controller connected to the PTZ camera.
  • the PTZ camera is driven to automatically rotate to the target angle.
  • the target angle is determined according to the position of the moving target in the image and the preset position, the target angle is the angle that makes the position of the moving target in the image the preset position, and the target angle includes the target angle in the horizontal direction and the target angle in the vertical direction. The target angle in the direction.
  • the target angle is determined according to the position information of the moving target, during the working process of the camera, if the position of the moving target changes, the camera will rotate to the new target angle determined according to the new position information, thereby realizing the control of the moving target.
  • Automatic tracking of the target exemplary, the moving object may be positioned in the center area of the image captured by the camera as the adjustment target, and the target angle may be determined according to the position information of the moving object, so that the moving object in the image is always located in the center area of the image. For example, when the imaging position of the moving object is lower, the camera can be controlled to rotate downward to offset the shooting range downward, so that the moving object is located in the center area of the image captured by the camera.
  • the moving target involved in the embodiment of the present application may be a person, that is, a user, a pet, or a robot, such as a cleaning robot.
  • the moving target involved in the embodiments of the present application is a human face.
  • the pan-tilt camera in the embodiments of the present application is used to automatically track and shoot a human face.
  • FIG. 6 is a schematic diagram of some possible camera configurations exemplarily shown in this application.
  • the PTZ camera supports the linux system, and may specifically include: an image acquisition module, a first processing module, a second processing module, and an encoding module and PTZ control module.
  • the image collection module starts to collect the image data stream, and outputs the collected image data stream to the first processing module and the second processing module respectively.
  • the first processing module processes the received image data stream and transmits it to the encoding module.
  • the preview data is displayed on the display; the second processing module processes the received image data stream to identify the data in the image.
  • the moving target is determined, the position information of the moving target is determined, and the position information of the moving target is output to the PTZ control module.
  • the PTZ control module determines the target angle according to the position information of the moving target, and adjusts the angle of the camera to the target angle.
  • the PTZ camera is only involved in capturing images and rotation angles, while image processing and encoding.
  • the second processing module includes a pre-trained neural network model, takes image data of uniform size as the input of the neural network model, runs the neural network model through the DSP to process the input image data, and outputs The location information of the moving object in the image. It should be understood that by using the sample sets corresponding to different moving objects to train the initial model, a neural network model for identifying different moving objects can be obtained.
  • the second processing module may include one or more processing units, respectively corresponding to one or more neural network models.
  • the second processing module may include an outline recognition unit and a face recognition unit, the outline recognition unit first identifies the user's limb outline from the image data, and then the facial recognition unit further recognizes the user according to the recognition result of the outline recognition unit. face, and finally output the position information of the moving target face.
  • the location of the moving object in the image can be represented by specifying the location of the keypoints in the image.
  • the position of the designated key point in the image does not match the preset position and/or the distance between the two is sufficiently large (eg, greater than a preset threshold)
  • the position of the moving object in the image may be specifically determined or represented according to the position coordinates of the pixel points of the moving object in the image in the specified coordinate system.
  • a plane rectangular coordinate system can be constructed with the upper left corner of the image as the origin and the right and downward directions as the positive directions, then each pixel in the image can pass through this rectangular coordinate system
  • the position information of the moving target is the coordinates of the corresponding pixel in the coordinate system.
  • the viewing angles of the cameras in the horizontal and vertical directions are HFOV and VFOV, respectively, and the viewing angles can be obtained according to the camera's CameraInfo.
  • the camera preview image supports 1080P, the width is 1920, and the height is 1080 pixels, then the image
  • the position of each pixel in can be (x, y), where the value range of x is (0, 1920); the value range of y is (0, 1080).
  • the key points of the user's limb or the position information of the limb frame in the image are used as the user's position information in the image.
  • Keypoints can refer to a series of points in a human image that can represent human features. For example, eyes, ears, nose, neck, shoulders, elbows, wrists, waist, knees, ankles, etc.
  • the number of key points can be set to multiple, and in one recognition process, all or part of the multiple key points need to be extracted, so as to determine the outer frame area wrapping the limb.
  • the key points may include 18, namely 2 eye points, 2 ear points, 1 nose point, 1 neck point, 2 shoulder points, 2 elbow points, 2 Wrist points, 2 waist points (or hip points), 2 knee points, and 2 ankle points.
  • the positions of some key points will change.
  • the relative position of the human body in the image captured by the camera will also change. For example, when the human body moves to the left, the position of the human body in the image captured by the camera will be shifted to the left.
  • the keypoints used to represent the user are referred to as designated keypoints, such as keypoints determined based on the user's facial features, such as nose points.
  • the user position may be represented by key point coordinates, such as nose point (x, y). Since the camera can be rotated in both horizontal and vertical directions, after calculating the (x, y) coordinates of the nose point, the x-axis and y-axis coordinates can be judged to determine whether the x-axis coordinates of the nose point are The center position of the entire image in the horizontal direction, and whether the y-axis coordinate of the nose point is located in the center position of the entire image in the vertical direction.
  • key point coordinates such as nose point (x, y). Since the camera can be rotated in both horizontal and vertical directions, after calculating the (x, y) coordinates of the nose point, the x-axis and y-axis coordinates can be judged to determine whether the x-axis coordinates of the nose point are The center position of the entire image in the horizontal direction, and whether the y-axis coordinate of the nose point is located in the center position of the entire image in the vertical direction.
  • judge the x-axis coordinate of the nose point when the resolution of the proofreading image is (1920, 1080), the x-axis coordinate of the center position of the proofreading image is 960; when it is judged that the x-axis coordinate of the nose point does not match 960 Or when the difference is large enough, it means that the user is not at the center of the entire image in the horizontal direction.
  • the y-axis coordinate of the nose point is judged: when the resolution of the proofreading image is (1920,1080), the y-axis coordinate of the center position of the proofreading image is 540; when it is judged that the y-axis coordinate of the nose point does not match 540 or When the difference is large enough, it means that the user is not at the center of the entire image in the vertical direction. At this time, it is determined that the angle of the camera needs to be adjusted, and the y-axis coordinate of the nose point and the center position of the proofreading image need to be determined in the vertical direction. on the target angle.
  • the distance between the user's position and the center point of the image can be calculated first; then, according to the calculated distance, the rotation angle is obtained by combining the maximum viewing angle of the camera and the image size; finally, the calculated rotation angle is sent in the form of a control command to
  • the pan-tilt control module enables the pan-tilt control module to control the pan-tilt motor to drive each rotating shaft to rotate, thereby adjusting the angle of the lens assembly.
  • the target angle in the horizontal direction can be calculated. Then, the user position is compared with the coordinate value of the center of the image, and the orientation of the user position relative to the center position is determined, thereby determining the rotation direction of the camera. That is, if the horizontal coordinate of the user's position is greater than the horizontal coordinate of the center position, turn the camera to the right; otherwise, turn the camera to the left.
  • the target angle in the horizontal direction can be determined by comparing the vertical direction difference between the user position and the image center position.
  • the method is the same as the adjustment method in the horizontal direction, and will not be repeated here.
  • a preset anti-shake algorithm is used to ensure the stability of the image.
  • the preset image stabilization algorithms include but are not limited to electronic image stabilization, optical image stabilization, and AI image stabilization.
  • FIG. 11 exemplarily shows another possible camera configuration diagram.
  • the PTZ camera supports the linux system, and may specifically include: an image acquisition module, a first processing module, a second processing module modules, encoding modules and communication modules.
  • the second encoding module obtains the processing result data (that is, the position information of the moving object)
  • the second encoding module encodes it
  • the communication module encodes the encoded moving object’s data.
  • the position information is sent to the display device controller, and the display device controller determines the target angle according to the position information of the moving target, and adjusts the angle of the camera to the target angle.
  • the image collection module starts to collect the image data stream, and outputs the collected image data stream to the first processing module and the second processing module respectively.
  • the first processing module processes the received image data stream to obtain preview data for output to the controller for display on the display, and outputs the obtained preview data to the first encoding module, and after encoding by the first encoding module , the preview data is output to the controller;
  • the second processing module processes the received image data stream to identify the moving target in the image, and determines the position information of the moving target, and outputs the position information of the moving target to the second encoding module.
  • the position information of the moving target is sent to the controller by the communication module.
  • the first encoding module and the second encoding module may be the same encoding module.
  • the communication module can be an RPC service (Remote Procedure Call Protocol), and the RPC service client transmits the output result of the second encoding module to the display device controller through the USB Network Gate network channel.
  • RPC service Remote Procedure Call Protocol
  • the RPC service on the controller distributes the received data to multiple applications, and the applications process it according to the needs.
  • the image data is sent to a controller connected to it.
  • the controller at least includes a first processing module, a second processing module, an encoding module, and a communication module.
  • the controller analyzes the image collected by the camera, determines the position information of the moving target, and when the position of the moving target in the image deviates from the preset position, controls the camera to rotate to the target angle, the target angle is to make the moving target
  • the position in the image is the angle of the preset position.
  • Some embodiments of the present application provide some display devices, which include a display, a camera, and a controller connected to the PTZ camera in the embodiments shown in FIGS. 1-7 .
  • the pan-tilt camera detects the moving objects in the collected images in real time, analyzes the position information of the moving objects, and sends the position information of the moving objects to the controller; when the controller determines the position of the moving objects in the images
  • the controller controls the pan-tilt camera to automatically rotate to the target angle.
  • the target angle is determined according to the position of the moving target in the image and the preset position.
  • the target angle is the position of the moving target in the image.
  • the angle of the preset position, the target angle includes the target angle in the horizontal direction and the target angle in the vertical direction.
  • an application installed on the display device may be used to process the image captured by the camera into a photo or video with special effect information.
  • the image data is collected through the camera, the image data and special effect data are received through the freambuffer object, and finally the image data and special effect data in the freambuffer object are drawn and displayed on the screen, and a picture file or video containing special effects is generated at the same time. document.
  • the special effect data may be provided by the application.
  • photos or videos with mirror effect may be captured by the camera, and these scenarios are referred to as mirror recording scenarios.
  • the camera data is mirror-processed data with a mirror effect, so it can be called mirror camera data.
  • the application installed on the display device can still be used to take photos or videos with special effects information.
  • the image data is collected through the camera, the camera data and special effect data are received through the freambuffer object, and then the image data and special effect data in the freambuffer object are mirrored through the mirror processing module, and finally the camera data and special effect data after mirroring are processed.
  • the special effect data is drawn and displayed on the screen, and a picture file or video file containing the special effect information is generated at the same time.
  • the mirrored camera data and the special effect data are mirrored at the same time. Therefore, the picture captured by the camera will be displayed in the forward direction, but the special effect information will be mirrored. Or, in order to display the mirrored camera picture, mirroring processing is performed on the forward camera data and the special effect data at the same time. Therefore, the picture and special effect information collected by the camera are mirrored and displayed at the same time.
  • FIG. 13 exemplarily shows the display effect of the text special effect before being mirrored and the display effect after being mirrored.
  • the embodiments of the present application provide some data processing methods, which are applied to display devices.
  • the method processes camera data or special effect data. It is to prevent the special effect information from being mirrored and displayed.
  • the data processing method provided by the embodiments of the present application may include the steps shown in FIG. 14 :
  • Step 11 acquire camera data and mirror effect data.
  • the camera data is the original image data collected by the camera
  • the mirror effect data is the data obtained after the forward special effect data is processed by mirroring. For example, during mirror recording, you can create a dreambuffer object to receive special effect data, and create a ScreenFilter to mirror the special effect data to obtain the mirror image data of the special effect data. Effect data may be provided by the application.
  • Step 12 performing mirror image processing on the camera data and the mirror image effect data to obtain mirror image camera data and forward special effect data;
  • Step 13 generate video file according to described mirror image camera data and forward special effect data
  • step 14 according to the mirroring camera data and the forward special effect data, the mirroring camera picture and the forward special effect picture superimposed on the mirroring camera picture are displayed.
  • the forward camera picture and the forward special effect picture are respectively opposite to the mirror camera picture and the mirror special effect picture.
  • the forward special effect picture refers to the picture presented by the special effect information without being mirrored and reversed.
  • the data processing method provided by the embodiments of the present application may include the following steps:
  • Step 21 acquiring mirror camera data and mirror special effect data.
  • the mirror effect data is the data obtained after the forward effect data is processed by mirroring. For example, during mirror recording, you can create a dreambuffer object to receive special effect data, and create a ScreenFilter to mirror the special effect data to obtain the mirror image data of the special effect data. Effect data may be provided by the application.
  • Step 22 performing mirror image processing on the mirrored camera data and the mirrored special effect data to obtain forward camera data and forward special effect data;
  • Step 23 generate video file according to described forward camera data and forward special effect data
  • Step 24 Display the forward camera picture and the forward special effect picture superimposed on the forward camera picture according to the forward camera data and the forward special effect data.
  • mirror special effect data is obtained by carrying out mirror image processing to the special effect data, so that the directions of the camera data and the special effect data are consistent, and the consistency of the camera picture and the special effect picture is guaranteed.
  • the data processing method provided by the embodiments of the present application may include the following steps:
  • Step 31 acquiring mirror camera data and special effect data
  • Step 32 processing the mirror camera data to obtain forward camera data.
  • Step 33 generate a video file according to the forward camera data and the special effect data
  • step 34 according to the forward camera data and the special effect data, the forward camera picture and the forward special effect picture superimposed on the forward camera picture are displayed.
  • the forward camera data is obtained by mirroring the mirror camera data, so that the directions of the camera data and the special effect data are consistent, thereby ensuring the consistency of the camera picture and the special effect picture.
  • the data processing method provided by the embodiments of the present application may include the following steps:
  • Step 41 obtaining mirror camera data and special effect data
  • Step 42 processing the mirror camera data to obtain forward camera data.
  • Step 43 generate a video file according to the forward camera data and the special effect data
  • step 44 according to the mirrored camera data and the special effect data, the mirrored camera picture and the forward special effect picture superimposed on the mirrored camera picture are displayed.
  • the mirror camera data is mirrored to obtain the forward camera data, which is used to generate a video file, and the mirror camera data and special effect data are used to display the preview screen, so that the camera screen is displayed.
  • the mirror image is displayed, and the special effect information is displayed in the forward direction.
  • the present invention also provides some non-volatile computer storage media, wherein the computer storage medium can store a program, and when the program is executed, it can include various embodiments of the screen saver display method and the screen saver jump method provided by the present invention some or all of the steps in .
  • the storage medium can be a magnetic disk, an optical disk, a read-only memory (English: read-only memory, abbreviated as: ROM) or a random access memory (English: random access memory, abbreviated as: RAM) and the like.
  • the technology in the embodiments of the present invention can be implemented by means of software plus a necessary general hardware platform.
  • the technical solutions in the embodiments of the present invention may be embodied in the form of software products in essence or the parts that make contributions to the prior art, and the computer software products may be stored in a storage medium, such as ROM/RAM , magnetic disk, optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.
  • a computer device which may be a personal computer, a server, or a network device, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本申请涉及显示设备和摄像头控制方法，云台摄像头在采集图像时，实时检测采集到的图像中的运动目标，并分析运动目标的位置信息，控制器在当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度，从而实现对运动目标的自动跟踪拍摄。

Description

摄像头、显示设备和摄像头控制方法
本申请要求在2020年8月21日提交中国专利局、申请号为202010852423.1、申请名称为“云台摄像头、显示设备和摄像头控制方法”的中国专利申请的优先权，和在2020年8月21日提交中国专利局、申请号为202010851657.4、申请名称为“显示设备和摄像头控制方法”的中国专利申请的优先权，以及在2020年8月21日提交中国专利局、申请号为202010852490.3、申请名称为“数据处理方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及摄像头技术领域，尤其涉及一种摄像头、显示设备和摄像头控制方法。
背景技术
显示设备可以为用户提供诸如音频、视频、图片等播放画面。如今，显示设备不仅可以为用户提供通过数据广播接收的直播电视节目内容，而且可以为用户提供诸如网络视频、网络游戏等各种应用和服务内容。
为了进一步满足用户的个性化需求，相关技术在显示设备上配置摄像头，通过显示设备的控制器获取摄像头采集的本地图像数据并进行处理、显示，在显示设备上实现视频聊天、拍照、录像等功能。
发明内容
本申请提供一些摄像头、显示设备和摄像头控制方法。
第一方面，本申请提供一些显示设备，包括：
显示器，用于显示用户界面；
云台摄像头，用于采集图像数据，根据所述图像数据识别图像中的运动目标及所述运动在图像中的位置信息；
控制器，用于当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
第二方面，本申请提供一些摄像头控制方法，应用于显示设备，所述方法包括：
从摄像头端获取运动目标在图像中的位置信息，所述运动目标在图像中的位置信息是由摄像头在采集到图像数据后处理所述图像数据得到的；
当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
第三方面，本申请提供一些云台摄像头，包括：
图像采集模块，用于采集图像数据；
第一处理模块，用于对所述图像数据进行处理，得到用于在显示设备的显示器上显示的预览数据；
第二处理模块，用于根据所述图像数据识别图像中的运动目标及所述运动在图像中的 位置；
云台控制模块，用于当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
第四方面，本申请还提供一些显示设备，包括：显示器、控制器和第一方面所述的云台摄像头，所述控制器与所述云台摄像头连接。
第五方面，本申请还提供一些摄像头控制方法，所述方法包括：
获取摄像头采集的图像数据；
根据所述图像数据识别图像中的运动目标及所述运动在图像中的位置；
当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
第五方面，本申请提供一些数据处理方法，所述方法包括：
获取摄像头数据和镜像特效数据；
对所述摄像头数据和镜像特效数据进行镜像处理，得到镜像摄像头数据和正向特效数据；
根据所述镜像摄像头数据和正向特效数据生成视频文件；
根据镜像摄像头数据和正向特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
第六方面，本申请提供一些数据处理方法，所述方法包括：
获取镜像摄像头数据和镜像特效数据；
对所述镜像摄像头数据和镜像特效数据进行镜像处理，得到正向摄像头数据和正向特效数据；
根据所述正向摄像头数据和正向特效数据生成视频文件；
根据正向摄像头数据和正向特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
第七方面，本申请提供一些数据处理方法，所述方法包括：
获取镜像摄像头数据和特效数据；
处理所述镜像摄像头数据得到正向摄像头数据；
根据所述正向摄像头数据和所述特效数据生成视频文件；
根据所述正向摄像头数据和所述特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
第八方面，本申请提供一些数据处理方法，所述方法包括：
获取镜像摄像头数据和特效数据；
处理所述镜像摄像头数据得到正向摄像头数据；
根据所述正向摄像头数据和所述特效数据生成视频文件；
根据所述镜像摄像头数据和所述特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
附图说明
为了更清楚地说明本申请的实施方式，下面将对实施例中所需要使用的附图作简单地介绍，显而易见地，对于本领域普通技术人员而言，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1中示例性示出了根据一些实施例的显示设备与控制装置之间操作场景的示意图；
图2中示例性示出了根据一些实施例的显示设备200的硬件配置框图；
图3中示例性示出了根据一些实施例的控制设备100的硬件配置框图；
图4为本申请实施例中显示设备与摄像头布置结构示意图；
图5a至图5f为本申请示例性示出的摄像头所处角度示意图；
图6为本申请示例性示出的一种可能的摄像头配置示意图；
图7为本申请实施例中像素点位置示意图；
图8为本申请实施例中关键点示意图；
图9为本申请实施例中计算旋转角度过程的几何关系示意图；
图10为本申请实施例中调整旋转角度过程示意图；
图11为本申请示例性示出的一种可能的摄像头配置示意图；
图12a为本申请示例性示出的一种拍摄具有特效信息的照片或者视频的实现方式示意图；
图12b为本申请示例性示出的另一种拍摄具有特效信息的照片或者视频的实现方式示意图；
图13示例性示出了文字特效被镜像处理前的显示效果和被镜像处理后的显示效果；
图14示例性示出了本申请实施例中的数据处理方法流程图。
具体实施方式
为使本申请的目的和实施方式更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施方式进行清楚、完整地描述，显然，描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。
需要说明的是，本申请中对于术语的简要说明，仅是为了方便理解接下来描述的实施方式，而不是意图限定本申请的实施方式。除非另有说明，这些术语应当按照其普通和通常的含义理解。
本申请中说明书和权利要求书及上述附图中的术语“第一”、“第二”、“第三”等是用于区别类似或同类的对象或实体，而不必然意味着限定特定的顺序或先后次序，除非另外注明。应该理解这样使用的用语在适当情况下可以互换。
术语“包括”和“具有”以及他们的任何变形，意图在于覆盖但不排他的包含，例如，包含了一系列组件的产品或设备不必限于清楚地列出的所有组件，而是可包括没有清楚地列出的或对于这些产品或设备固有的其它组件。
术语“模块”是指任何已知或后来开发的硬件、软件、固件、人工智能、模糊逻辑或硬件或/和软件代码的组合，能够执行与该元件相关的功能。
图1为根据实施例中显示设备与控制装置之间操作场景的示意图。如图1所示， 用户可通过智能终端300或控制装置100操作显示设备200。
控制装置100可以是遥控器，遥控器和显示设备的通信包括红外协议通信或蓝牙协议通信，及其他短距离通信方式，通过无线或有线方式来控制显示设备200。用户可以通过遥控器上按键、语音输入、控制面板输入等输入用户指令，来控制显示设备200。
在一些实施例中，也可以使用智能终端300(如移动终端、平板电脑、计算机、笔记本电脑等)以控制显示设备200。例如，使用在智能设备上运行的应用程序控制显示设备200。
在一些实施例中，显示设备200还可以采用除了控制装置100和智能设备300之外的方式进行控制，例如，可以通过显示设备200设备内部配置的获取语音指令的模块直接接收用户的语音指令控制，也可以通过显示设备200设备外部设置的语音控制设备来接收用户的语音指令控制。
在一些实施例中，显示设备200还与服务器400进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器400可以向显示设备200提供各种内容和互动。
图2示例性示出了根据示例性实施例中控制装置100的配置框图。如图2所示，控制装置100包括控制器110、通信接口130、用户输入/输出接口140、存储器、供电电源。控制装置100可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起用用户与显示设备200之间交互中介作用。
图3示出了根据示例性实施例中显示设备200的硬件配置框图。
显示设备200包括调谐解调器210、通信器220、检测器230、外部装置接口240、控制器250、显示器275、音频输出接口285、存储器、供电电源、用户接口中的至少一些。
显示器275包括用于呈现画面的显示屏组件，以及驱动图像显示的驱动组件，用于接收源自控制器输出的图像信号，进行显示视频内容、图像内容以及菜单操控界面的组件以及用户操控UI界面。
显示器275可为液晶显示器、OLED显示器、以及投影显示器，还可以为一些投影装置和投影屏幕。
通信器220是用于根据各种通信协议类型与外部设备或服务器进行通信的组件。例如：通信器可以包括Wifi模块，蓝牙模块，有线以太网模块等其他网络通信协议芯片或近场通信协议芯片，以及红外接收器中的至少一些。显示设备200可以通过通信器220与外部控制设备100或服务器400建立控制信号和数据信号的发送和接收。
用户接口，可用于接收控制装置100(如：红外遥控器等)的控制信号。
检测器230用于采集外部环境或与外部交互的信号。例如，检测器230包括光接收器，用于采集环境光线强度的传感器；或者，检测器230包括图像采集器，如摄像头，可以用于采集外部环境场景、用户的属性或用户交互手势，再或者，检测器230包括声音采集器，如麦克风等，用于接收外部声音。
外部装置接口240可以包括但不限于如下：高清多媒体接口接口(HDMI)、模拟或数据高清分量输入接口(分量)、复合视频输入接口(CVBS)、USB输入接口(USB)、RGB端口等任一个或多个接口。也可以是上述多个接口形成的复合性的输入/输出接口。
控制器250和调谐解调器210可以位于不同的分体设备中，即调谐解调器210也可在控制器250所在的主体设备的外置设备中，如外置机顶盒等。
控制器250，通过存储在存储器上中各种软件控制程序，来控制显示设备的工作和响应用户的操作。控制器250控制显示设备200的整体操作。例如：响应于接收到用于选择在显示器275上显示UI对象的用户命令，控制器250便可以执行与由用户命令选择的对象有关的操作。
对象可以是可选对象中的任何一个，例如超链接、图标或其他可操作的控件。与所选择的对象有关操作有：显示连接到超链接页面、文档、图像等操作，或者执行与所述图标相对应程序的操作。
在一些实施例中，用户可在显示器275上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
“用户界面”可以指应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(Graphic User Interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导航栏、Widget等可视的界面元素。在一些实施例中，系统可以包括内核(Kernel)、命令解析器(shell)、文件系统和应用程序。内核、shell和文件系统一起组成了基本的操作系统结构，它们让用户可以管理文件、运行程序并使用系统。上电后，内核启动，激活内核空间，抽象硬件、初始化硬件参数等，运行并维护虚拟内存、调度器、信号及进程间通信(IPC)。内核启动后，再加载Shell和用户应用程序。应用程序在启动后被编译成机器码，形成一个进程。
本申请实施例中，如图4所示，摄像头作为一种检测器可以内置或外接显示设备上，在启动运行后，摄像头可以检测图像数据。摄像头可以通过接口部件与控制器连接，从而将检测的图像数据发送给控制器进行处理。摄像头所处的角度，如其在竖直方向上的俯仰角度或者其在水平方向上的角度，关乎摄像头的视场范围。云台是安装和固定摄像头的支撑装置，分为固定云台和电动云台，其中，电动云台适用于大范围扫描拍摄，它可以扩大摄像头的视场范围。电动云台可以是只能左右旋转的水平旋转云台，也可以是既能左右旋转又能上下旋转的全方位云台。一般，全方位云台内安装两个电动机，分别用于驱动云台在水平方向和垂直方向上旋转，以改变摄像头所处的角度。
云台摄像头，包括镜头组件和云台组件。其中，镜头组件可以是基于CCD(Charge Coupled Device，电荷耦合器件)或CMOS(Complementary Metal Oxide Semiconductor，互补金属氧化物半导体)的检测原理，以根据用户图像生成电信号的图像数据。对于配置有云台摄像头的显示设备，利用显示设备控制器对电动云台进行控制，可以使摄像头在多个角度下进行拍摄。
需要说明的是，本申请中的摄像头可实现采集图像相关的所有功能，在不涉及角度旋 转时，与不具有角度调整的摄像头不作区分。当可调整采集图像的角度时，在没有特别指出的情况下，为云台摄像头。此外，凡是能实现本文中所述功能的摄像头均在本申请的范围之类，并不对摄像头的结构和组成构件做特殊的限定。
摄像头在水平方向和/或垂直方向可旋转的极限角度，可以根据需求进行设计。示例性的，摄像头在水平方向上的可旋转角度范围可以为0°～120°，其中，0°和120°分别为在水平方向上的两个旋转方向(向左和向右)对应的极限角度；摄像头在垂直方向上的可旋转角度可以为0°～180°，其中，0°和180°分别为在垂直方向上的两个旋转方向(向上和向下)对应的极限角度。
图5a至图5f为本申请示例性示出的摄像头所处角度示意图，其中，图5a示例性示出了摄像头在垂直方向上的俯仰角度为0°时的状态，图5b示例性示出了摄像头在垂直方向上的俯仰角度为90°时的状态，图5c示例性示出了摄像头在垂直方向上的俯仰角度为105°时的状态，图5d示例性示出了摄像头在水平方向上的水平角度为0°时的状态，图5e示例性示出了摄像头在水平方向上的水平角度为60°时的状态，图5f示例性示出了摄像头在水平方向上的水平角度为120°时的状态。
本申请实施例提供一些显示设备，包括如图1-7所示实施例中的显示器、云台摄像头以及与云台摄像头连接的控制器。其中，在采集图像时，实时检测采集到的图像中的运动目标，并分析运动目标的位置信息，当运动目标在图像中的位置偏离预设位置时，驱动云台摄像头自动旋转到目标角度，该目标角度根据运动目标在图像中的位置和预设位置确定，该目标角度为使运动目标在图像中的位置为预设位置的角度，该目标角度包括在水平方向上的目标角度和在垂直方向上的目标角度。
应当理解，由于该目标角度根据运动目标的位置信息确定，因此在摄像头工作过程中，若运动目标的位置发生变化，摄像头将转动到根据新的位置信息确定的新的目标角度，进而实现对运动目标的自动追踪拍摄。示例性的，可以以使运动目标位于摄像头所拍摄图像的中心区域为调整目标，根据运动目标的位置信息确定目标角度，从而图像中的运动目标始终位于图像的中心区域。例如，当运动目标的成像位置偏下时，可以通过控制摄像头向下转动，使拍摄范围向下偏移，使运动目标位于摄像头所拍摄图像的中心区域。
需要说明的是，本申请实施例涉及的运动目标可以是人物，即用户，还可以是宠物，还可以是机器人，如扫地机器人等。
在一些实施例中，本申请实施例涉及的运动目标是人物面部。在这些实施例中，本申请实施例中的云台摄像头，用于对人脸进行自动追踪拍摄。
图6为本申请示例性示出的一些可能的摄像头配置示意图，如图6所示，云台摄像头支持linux系统，具体可以包括：图像采集模块、第一处理模块、第二处理模块、编码模块和云台控制模块。摄像头启动运动后，图像采集模块开始采集图像数据流，并将采集的图像数据流分别输出至第一处理模块和第二处理模块。第一处理模块对接收的图像数据流进行处理，传输至编码模块，经编码模块编码后，在显示器上进行显示的预览数据；第二处理模块对接收的图像数据流进行处理，以识别图像中的运动目标，并确定运动目标的位置 信息，将运动目标的位置信息输出至云台控制模块，云台控制模块根据运动目标的位置信息确定目标角度，将摄像头所处角度调整到目标角度。
在一些实施例中，云台摄像头仅涉及采集图像和旋转角度，而图像处理、编码。
在一些可能的实现方式中，第二处理模块包括预先训练好的神经网络模型，以尺寸统一的图像数据作为神经网络模型的输入，通过DSP运行神经网络模型以对输入的图像数据进行处理，输出运动目标在图像中的位置信息。应当理解，使用不同运动目标对应的样本集对初始模型进行训练，可以得到用于识别不同运动目标的神经网络模型。
需要说明的是，在一些实施例中，第二处理模块可以包括一个或者多个处理单元，分别对应于一个或者多个神经网络模型。示例性的，第二处理模块可以包括轮廓识别单元和面部识别单元，由轮廓识别单元首先从图像数据中识别出用户的肢体轮廓，再由面部识别单元根据轮廓识别单元的识别结果进一步识别出用户的面部，最终输出运动目标面部的位置信息。
在一些实施例中，运动目标在图像中的位置可以通过指定关键点在图像中的位置进行表示。在这些实施例中国，当指定关键点在图像中的位置与预设位置不符和/或二者之间的距离足够大(如大于预设阈值)时，确定运动目标在图像中的位置偏离预设位置。
在一些实施例中，运动目标在图像中的位置具体可以根据运动目标在图像中的像素点在指定坐标系中的位置坐标确定或表示。例如，可以根据摄像头的分辨率和可视角度，以图像的左上角为原点，以向右和向下为正方向构建平面直角坐标系，则图像中的各个像素点均能够通过这一直角坐标系进行表示，运动目标的位置信息即为其对应的像素点在该坐标系中的坐标。示例性的，如图7所示，水平方向和垂直方向摄像头可视角度分别为HFOV和VFOV，可视角度可以根据摄像头CameraInfo获取，摄像头预览图像支持1080P，宽度为1920，高度1080像素，则图像中每个像素点的位置都可以为(x，y)，其中x的取值范围为(0，1920)；y的取值范围为(0,1080)。
在运动目标为用户的实施场景中，为了能够准确表达用户在图像中的位置，以用户肢体的关键点或者肢体框在图像中的位置信息作为用户在图像中的位置信息。关键点可以是指人体图像中能够代表人体特征的一系列点。例如，眼睛、耳朵、鼻子、脖子、肩部、手肘、手腕、腰部、膝关节以及踝关节等。关键点的数量可以设置为多个，并且在一次识别过程中需要对多个关键点的全部或部分进行位置提取，从而确定包裹肢体的外框区域。例如，如图8所示，关键点可以包括18个，即2个眼睛点、2个耳朵点、1个鼻子点、1个脖子点、2个肩部点、2个肘部点、2个腕部点、2个腰部点(或臀部点)、2个膝关节点以及2个踝关节点。显然，当用户所处位置发生改变或者姿态发生变化时，部分关键点的位置将发生变化。随着这种变化的出现，摄像头采集的图像中人体相对位置也将发生变化。例如，当人体向左移动位置时，将使摄像头采集的图像中人体位置偏左。因此，在得到用户位置后，还需要对比用户位置与校对图像中的预设区域(如中心位置)，从而确定是否需要调整摄像头的角度，以及需要调整的目标角度。
在一些实施例中，用于代表用户的关键点称为指定关键点，如基于用户面部特征确定 的关键点，如鼻子点。
示例性的，用户位置可以通过关键点坐标进行表示，如鼻子点(x，y)。由于摄像头可以在水平和垂直两个方向上旋转，因此在计算获得鼻子点(x，y)坐标后，可以分别对x轴坐标和y轴坐标进行判断，以分别确定鼻子点的x轴坐标是否位于整个图像在水平方向上的中心位置，以及鼻子点的y轴坐标是否位于整个图像在垂直方向上的中心位置。示例性的，对鼻子点的x轴坐标进行判断：当校对图像为分辨率为(1920,1080)时，校对图像的中心位置x轴坐标为960；当判定鼻子点的x轴坐标与960不符或者相差足够大时，说明用户不位于整个图像在水平方向上的中心位置，此时确定需要调整摄像头的角度，并且需要根据脖子点的x轴坐标和校对图像的中心位置x轴坐标确定在水平方向上的目标角度。类似的，对鼻子点的y轴坐标进行判断：当校对图像为分辨率为(1920,1080)时，校对图像的中心位置y轴坐标为540；当判定鼻子点的y轴坐标与540不符或者相差足够大时，说明用户不位于整个图像在垂直方向上的中心位置，此时确定需要调整摄像头的角度，并且需要根据鼻子点的y轴坐标和校对图像的中心位置y轴坐标确定在垂直方向上的目标角度。
具体的，可以先计算用户位置和图像的中心点之间的距离；再根据计算的距离，结合摄像头的最大视角以及图像尺寸计算获得旋转角度；最后将计算的旋转角度以控制指令的形式发送给云台控制模块，使得云台控制模块控制云台电机带动各转轴进行转动，从而调整镜头组件的角度。
示例性的，如图9、图10所示，摄像头的预览分辨率为1920×1080，图像的水平宽度：imgWidth＝1920；图像水平中心位置坐标x＝960；用户位置坐标为(x0，y0)水平中心位置坐标为x0；水平视角为hfov；则用户位置与中心的距离为：hd＝x–x0，则摄像头在水平方向上的旋转角度则按照下式可计算获得：
Figure PCTCN2021100429-appb-000001
通过上式，可以计算出在水平方向上的目标角度。再对用户位置与图像中心的坐标数值进行比较，确定用户位置相对于中心位置的方位，从而确定摄像头的旋转方向。即，如果用户位置的水平坐标大于中心位置的水平坐标，则向右转动摄像头；反之向左转动摄像头。
上述实施例是以水平方向坐标为例进行判断、调整，在其他实施例中，通过比较用户位置与图像中心位置之间的竖直方向差异，可以确定在水平方向上的目标角度，具体的调整方法与水平方向的调整方法相同，此处不予赘述。
在一些实施例中，在调整摄像头转动的过程中，利用预置防抖算法来保证图像画面的稳定性。预置防抖算法包括但不限于电子防抖、光学防抖和AI防抖等。
基于上述实施例，图11示例性示出的另一种可能的摄像头配置示意图，如图11所示，云台摄像头支持linux系统，具体可以包括：图像采集模块、第一处理模块、第二处理模块、编码模块和通信模块。与图6所示实施例不同的是，第二编码模块得到处理结果数据(即运动目标的位置信息)后，由第二编码模块对其进行编码，并由通信模块将经编码的运动 目标的位置信息发送给显示设备控制器，由显示设备控制器根据运动目标的位置信息确定目标角度，将摄像头所处角度调整到目标角度。具体的，摄像头启动运动后，图像采集模块开始采集图像数据流，并将采集的图像数据流分别输出至第一处理模块和第二处理模块。第一处理模块对接收的图像数据流进行处理，以得到用于输出至控制器以在显示器上进行显示的预览数据，将得到的预览数据输出至第一编码模块，经第一编码模块编码后，预览数据被输出至控制器；第二处理模块对接收的图像数据流进行处理，以识别图像中的运动目标，并确定运动目标的位置信息，将运动目标的位置信息输出至第二编码模块，经第二编码模块编码后，运动目标的位置信息由通信模块发送给控制器。
在一些实施例中，第一编码模块和第二编码模块可以为同一个编码模块。
在一些实施例中，通信模块可以为RPC服务(Remote Procedure Call Protocol，远程过程调用协议)，RPC服务客户端通过USB Network Gate网络通道将第二编码模块的输出结果传输到显示设备控制器上的RPC服务，控制器上的RPC服务将接收到的数据分发给多个应用，由应用根据需求进行处理。
本申请的一些实施例中，当云台摄像头采集到图像时，将图像数据发送至与之相连的控制器。此时，控制器至少包括第一处理模块、第二处理模块、编码模块、通信模块。控制器对摄像头采集到的图像进行分析，确定运动目标的位置信息，以及当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
本申请的一些实施例中提供一些显示设备，该包括如图1-7所示实施例中的显示器、摄像头以及与云台摄像头连接的控制器。其中，云台摄像头在采集图像时，实时检测采集到的图像中的运动目标，并分析运动目标的位置信息，将运动目标的位置信息发送给控制器；当控制器确定运动目标在图像中的位置偏离预设位置时，由控制器控制云台摄像头自动旋转到目标角度，该目标角度根据运动目标在图像中的位置和预设位置确定，该目标角度为使运动目标在图像中的位置为预设位置的角度，该目标角度包括在水平方向上的目标角度和在垂直方向上的目标角度。
在一些实施场景中，可以利用安装在显示设备上的应用将摄像头采集的图像处理成具有特效信息的照片或者视频。如图12a所示，通过摄像头采集图像数据，通过freambuffer object接收图像数据和特效数据，最后将freambuffer object中的图像数据和特效数据绘制在屏幕上显示出来，同时生成包含特效效果的图片文件或者视频文件。其中，特效数据可由应用提供。
在一些实施场景中，可以通过摄像头拍摄具有镜像效果的照片或者视频，将这些场景称为镜像录制场景。在镜像录制场景中，摄像头数据是经镜像处理的数据，具有镜像效果，因此可以称为镜像摄像头数据。
在镜像录制场景中，仍可以利用安装在显示设备上的应用拍摄具有特效信息的照片或者视频。如图12b所示，通过摄像头采集图像数据，通过freambuffer object接收摄像头数据和特效数据，再通过镜像处理模块将freambuffer object中的图像数据和特效数据进行镜像处 理，最后经镜像处理后的摄像头数据和特效数据绘制在屏幕上显示出来，同时生成包含特效信息的图片文件或者视频文件。
值得注意的是，为了将镜像摄像头数据正向显示出来，对镜像摄像头数据和特效数据同时做了镜像处理，因此，摄像头采集的画面得到正向显示，但特效信息将被镜像显示。或者，为了显示出镜像的摄像头画面，对正向摄像头数据和特效数据同时做了镜像处理，因此，摄像头采集的画面和特效信息同时被镜像显示。
可以理解的是，当特效信息中的存在如文字等不圆等对称的信息时，由于其被镜像处理后将会被翻转显示，因此极易被用户察觉，不利于用户体验。示例性的，图13示例性示出了文字特效被镜像处理前的显示效果和被镜像处理后的显示效果。
为解决上述问题，本申请实施例提供一些数据处理方法，应用于显示设备，该方法在显示具有特效信息的摄像头画面或者生成具有特效信息的视频文件时，对摄像头数据或者特效数据进行处理，目的是防止特效信息被镜像显示出来。
在一些实施例中，本申请实施例提供的数据处理方法可以包括图14所示步骤：
步骤11，获取摄像头数据和镜像特效数据。
其中，摄像头数据即为由摄像头采集到的原始图像数据，镜像特效数据为正向特效数据经镜像处理后得到的数据。例如，在镜像录制时，可以新建一个freambuffer object以接收特效数据，并创建一个ScreenFilter，以对特效数据进行镜像处理，得到特效数据的镜像数据。特效数据可由应用提供。
步骤12，对所述摄像头数据和镜像特效数据进行镜像处理，得到镜像摄像头数据和正向特效数据；
步骤13，根据所述镜像摄像头数据和正向特效数据生成视频文件；
以及，步骤14，根据镜像摄像头数据和正向特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
需要说明的是，正向摄像头画面和正向特效画面分别与镜像摄像头画面和镜像特效画面相反，例如，正向特效画面是指特效信息未经镜像反转所呈现的画面。
在一些实施例中，本申请实施例提供的数据处理方法可以包括如下步骤：
步骤21，获取镜像摄像头数据和镜像特效数据。
镜像特效数据为正向特效数据经镜像处理后得到的数据。例如，在镜像录制时，可以新建一个freambuffer object以接收特效数据，并创建一个ScreenFilter，以对特效数据进行镜像处理，得到特效数据的镜像数据。特效数据可由应用提供。
步骤22，对所述镜像摄像头数据和镜像特效数据进行镜像处理，得到正向摄像头数据和正向特效数据；
步骤23，根据所述正向摄像头数据和正向特效数据生成视频文件；
步骤24，根据正向摄像头数据和正向特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
由以上实施例可以看出，在镜像录制场景中，通过将特效数据进行镜像处理得到镜像 特效数据，使摄像头数据和特效数据的方向一致，并保证摄像头画面与特效画面的一致性。
在一些实施例中，本申请实施例提供的数据处理方法可以包括如下步骤：
步骤31，获取镜像摄像头数据和特效数据；
步骤32，处理所述镜像摄像头数据得到正向摄像头数据。
步骤33，根据所述正向摄像头数据和所述特效数据生成视频文件；
以及在步骤34中，根据所述正向摄像头数据和所述特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
由以上实施例可以看出，在镜像录制场景中，通过将镜像摄像头数据进行镜像处理得到正向摄像头数据，使摄像头数据和特效数据的方向一致，从而保证摄像头画面与特效画面的一致性。
在一些实施例中，本申请实施例提供的数据处理方法可以包括如下步骤：
步骤41，获取镜像摄像头数据和特效数据；
步骤42，处理所述镜像摄像头数据得到正向摄像头数据。
步骤43，根据所述正向摄像头数据和所述特效数据生成视频文件；
以及在步骤44中，根据所述镜像摄像头数据和所述特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
由以上实施例可以看出，在镜像录制场景中，将镜像摄像头数据进行镜像处理得到正向摄像头数据，用于生成视频文件，将镜像摄像头数据和特效数据用于显示预览画面，从而将摄像头画面镜像显示出来，将特效信息正向显示出来。
具体实现中，本发明还提供一些非易失性计算机存储介质，其中，该计算机存储介质可存储有程序，该程序执行时可包括本发明提供的屏保展示方法和屏保跳转方法的各实施例中的部分或全部步骤。所述的存储介质可为磁碟、光盘、只读存储记忆体(英文：read-only memory，简称：ROM)或随机存储记忆体(英文：random access memory，简称：RAM)等。
本领域的技术人员可以清楚地了解到本发明实施例中的技术可借助软件加必需的通用硬件平台的方式来实现。基于这样的理解，本发明实施例中的技术方案本质上或者说对现有技术做出贡献的部分可以以软件产品的形式体现出来，该计算机软件产品可以存储在存储介质中，如ROM/RAM、磁碟、光盘等，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本发明各个实施例或者实施例的某些部分所述的方法。
最后应说明的是：以上各实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述各实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分或者全部技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的范围。
为了方便解释，已经结合具体的实施方式进行了上述说明。但是，上述示例性的讨论不是意图穷尽或者将实施方式限定到上述公开的具体形式。根据上述的教导，可 以得到多种修改和变形。上述实施方式的选择和描述是为了更好的解释原理以及实际的应用，从而使得本领域技术人员更好的使用所述实施方式以及适于具体使用考虑的各种不同的变形的实施方式。

Claims (12)

  1. 一种显示设备，包括：
    显示器，用于显示用户界面；
    摄像头，用于采集图像数据，根据所述图像数据识别图像中的运动目标及所述运动在图像中的位置信息；
    控制器，用于当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
  2. 根据权利要求1所述的显示设备，所述预设位置为所述图像的中心位置，所述运动目标在图像中的位置为指定关键点在图像中的位置；当所述指定关键点在图像中的位置与所述中心位置之间的距离大于预设阈值时，确定所述运动目标在图像中的位置偏离预设位置。
  3. 根据权利要求2所述的显示设备，所述运动目标为用户，所述指定关键点为基于用户面部特征确定的关键点。
  4. 一种摄像头控制方法，应用于显示设备，所述方法包括：
    从摄像头端获取运动目标在图像中的位置信息，所述运动目标在图像中的位置信息是由摄像头在采集到图像数据后处理所述图像数据得到的；
    当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
  5. 一种摄像头，包括：
    图像采集模块，用于采集图像数据；
    第一处理模块，用于对所述图像数据进行处理，得到用于在显示设备的显示器上显示的预览数据；
    第二处理模块，用于根据所述图像数据识别图像中的运动目标及所述运动在图像中的位置；
    云台控制模块，用于当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述目标角度为使运动目标在图像中的位置为所述预设位置的角度。
  6. 根据权利要求5所述的摄像头，所述预设位置为所述图像的中心位置，所述运动目标在图像中的位置为指定关键点在图像中的位置；当所述指定关键点在图像中的位置与所述中心位置之间的距离大于预设阈值时，确定所述运动目标在图像中的位置偏离预设位置。
  7. 根据权利要求6所述的摄像头，所述运动目标为用户，所述指定关键点为基于用户面部特征确定的关键点。
  8. 一种摄像头控制方法，所述方法包括：
    获取摄像头采集的图像数据；
    根据所述图像数据识别图像中的运动目标及所述运动在图像中的位置；
    当所述运动目标在图像中的位置偏离预设位置时，控制摄像头旋转到目标角度，所述 目标角度为使运动目标在图像中的位置为所述预设位置的角度。
  9. 一种数据处理方法，所述方法包括：
    获取摄像头数据和镜像特效数据；
    对所述摄像头数据和镜像特效数据进行镜像处理，得到镜像摄像头数据和正向特效数据；
    根据所述镜像摄像头数据和正向特效数据生成视频文件；
    根据镜像摄像头数据和正向特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
  10. 一种数据处理方法，所述方法包括：
    获取镜像摄像头数据和镜像特效数据；
    对所述镜像摄像头数据和镜像特效数据进行镜像处理，得到正向摄像头数据和正向特效数据；
    根据所述正向摄像头数据和正向特效数据生成视频文件；
    根据正向摄像头数据和正向特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
  11. 一种数据处理方法，所述方法包括：
    获取镜像摄像头数据和特效数据；
    处理所述镜像摄像头数据得到正向摄像头数据；
    根据所述正向摄像头数据和所述特效数据生成视频文件；
    根据所述正向摄像头数据和所述特效数据，显示出正向摄像头画面和叠加在正向摄像头画面上的正向特效画面。
  12. 一种数据处理方法，所述方法包括：
    获取镜像摄像头数据和特效数据；
    处理所述镜像摄像头数据得到正向摄像头数据；
    根据所述正向摄像头数据和所述特效数据生成视频文件；
    根据所述镜像摄像头数据和所述特效数据，显示出镜像摄像头画面和叠加在镜像摄像头画面上的正向特效画面。
PCT/CN2021/100429 2020-08-21 2021-06-16 摄像头、显示设备和摄像头控制方法 WO2022037215A1 (zh)

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202010852490.3 2020-08-21
CN202010852423.1 2020-08-21
CN202010851657.4 2020-08-21
CN202010851657 2020-08-21
CN202010852423 2020-08-21
CN202010852490 2020-08-21

Publications (1)

Publication Number Publication Date
WO2022037215A1 true WO2022037215A1 (zh) 2022-02-24

Family

ID=80322571

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/100429 WO2022037215A1 (zh) 2020-08-21 2021-06-16 摄像头、显示设备和摄像头控制方法

Country Status (1)

Country Link
WO (1) WO2022037215A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115598989A (zh) * 2022-08-29 2023-01-13 南通逸飞智能科技有限公司（Cn） 基于人工智能技术的智能家居控制主机

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104778726A (zh) * 2015-04-29 2015-07-15 深圳市保千里电子有限公司 一种基于人体特征进行运动轨迹跟踪的方法及系统
CN104966318A (zh) * 2015-06-18 2015-10-07 清华大学 具有图像叠加和图像特效功能的增强现实方法
US20150350606A1 (en) * 2014-05-29 2015-12-03 Abdullah I. Khanfor Automatic object tracking camera
CN105898136A (zh) * 2015-11-17 2016-08-24 乐视致新电子科技（天津）有限公司 摄像视角调节方法、系统及电视
CN108848405A (zh) * 2018-06-29 2018-11-20 广州酷狗计算机科技有限公司 图像处理方法和装置
CN111093050A (zh) * 2018-10-19 2020-05-01 浙江宇视科技有限公司 一种目标监控方法及装置

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150350606A1 (en) * 2014-05-29 2015-12-03 Abdullah I. Khanfor Automatic object tracking camera
CN104778726A (zh) * 2015-04-29 2015-07-15 深圳市保千里电子有限公司 一种基于人体特征进行运动轨迹跟踪的方法及系统
CN104966318A (zh) * 2015-06-18 2015-10-07 清华大学 具有图像叠加和图像特效功能的增强现实方法
CN105898136A (zh) * 2015-11-17 2016-08-24 乐视致新电子科技（天津）有限公司 摄像视角调节方法、系统及电视
CN108848405A (zh) * 2018-06-29 2018-11-20 广州酷狗计算机科技有限公司 图像处理方法和装置
CN111093050A (zh) * 2018-10-19 2020-05-01 浙江宇视科技有限公司 一种目标监控方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115598989A (zh) * 2022-08-29 2023-01-13 南通逸飞智能科技有限公司（Cn） 基于人工智能技术的智能家居控制主机
CN115598989B (zh) * 2022-08-29 2024-02-09 广东韩江智能科技有限公司 基于人工智能技术的智能家居控制主机

Similar Documents

Publication Publication Date Title
WO2022001407A1 (zh) 一种摄像头的控制方法及显示设备
US11189055B2 (en) Information processing apparatus and method and program
US9811910B1 (en) Cloud-based image improvement
JP6058978B2 (ja) 画像処理装置及び画像処理方法、撮影装置、並びにコンピューター・プログラム
US20180367732A1 (en) Visual cues for managing image capture
JP5967473B2 (ja) 撮像装置及び撮像システム
US8638372B2 (en) Image capture unit with changeable image capture direction
US20150208032A1 (en) Content data capture, display and manipulation system
JP2005100084A (ja) 画像処理装置及び方法
JP2014127001A (ja) 画像処理システム、画像処理方法およびプログラム
CN113655887B (zh) 一种虚拟现实设备及静态录屏方法
WO2022037535A1 (zh) 显示设备及摄像头追踪方法
US9743048B2 (en) Imaging apparatus, camera unit, display unit, image-taking method, display method and computer readable recording medium recording program thereon
US20170052600A1 (en) Using natural movements of a hand-held device to manipulate digital content
WO2022001406A1 (zh) 一种显示方法及显示设备
US10462358B2 (en) Information processing apparatus, information processing system, and information processing method
JP2013013062A (ja) 撮像装置及び撮像システム
US20220070412A1 (en) Communication terminal, image communication system, method of displaying image, and recording medium
WO2018121401A1 (zh) 一种全景视频图像的拼接方法及全景相机
JP2015046171A (ja) 画像を生成する装置および方法
CN112672062A (zh) 一种显示设备及人像定位方法
WO2023040288A1 (zh) 显示设备及设备控制方法
WO2022037215A1 (zh) 摄像头、显示设备和摄像头控制方法
CN113473024A (zh) 显示设备、云台摄像头和摄像头控制方法
CN112929750B (zh) 一种摄像头调节方法及显示设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21857312

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21857312

Country of ref document: EP

Kind code of ref document: A1