WO2022100262A1 - 显示设备、人体姿态检测方法及应用 - Google Patents

显示设备、人体姿态检测方法及应用 Download PDF

Info

Publication number
WO2022100262A1
WO2022100262A1 PCT/CN2021/117797 CN2021117797W WO2022100262A1 WO 2022100262 A1 WO2022100262 A1 WO 2022100262A1 CN 2021117797 W CN2021117797 W CN 2021117797W WO 2022100262 A1 WO2022100262 A1 WO 2022100262A1
Authority
WO
WIPO (PCT)
Prior art keywords
user
image
human body
target
frame image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2021/117797
Other languages
English (en)
French (fr)
Inventor
刘俊秀
姜俊厚
刘晋
李佳琳
祝欣培
李保成
高伟
岳国华
初德进
杨鲁明
鲍姗娟
王大勇
于文钦
王之奎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hisense Visual Technology Co Ltd
Original Assignee
Hisense Visual Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from CN202011262476.4A external-priority patent/CN112543359B/zh
Priority claimed from CN202011267208.1A external-priority patent/CN112473121B/zh
Priority claimed from CN202110960378.6A external-priority patent/CN116246333A/zh
Application filed by Hisense Visual Technology Co Ltd filed Critical Hisense Visual Technology Co Ltd
Priority to CN202180057495.XA priority Critical patent/CN116114250B/zh
Publication of WO2022100262A1 publication Critical patent/WO2022100262A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223—Cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/485—End-user interface for client configuration

Definitions

  • the present application relates to display device technology, and in particular, to a display device, a human body posture detection method, and applications.
  • display devices are getting richer and richer, and more display devices are equipped with image acquisition devices such as cameras, which acquire user images through cameras, and cooperate with the "limb movement" application to enable display devices to display user body images in real time.
  • image acquisition devices such as cameras
  • limb movement application to enable display devices to display user body images in real time.
  • the application will also display the changed image, and detect and correct the shape of the body movement through the detection program to achieve the effect of exercise guidance.
  • Some embodiments of the present application provide a display device, including: a camera for capturing a target user and background images in a dodgeball game in a detection area; a display for displaying a dodgeball game including the target user and background images a user interface; a controller configured to: control the user interface to be at the first position according to the first size of the target user in the background image and the first position of the target user in the user interface Displaying a first sign, wherein the first sign is used to trigger the launch of the dodge ball, and the size of the first sign corresponds to the first size; when the upper torso of the target user is not completely located in the collection area, And when the target user is leaving the collection area, the controller controls the user interface to no longer display the first logo.
  • Some embodiments of the present application provide an application for human posture detection, including: controlling a target user according to a first size of a target user in a background image of a user interface of a dodgeball game and a first position of the target user in the user interface
  • the user interface displays a first logo at the first position, wherein the first logo is used to trigger dodgeball launch, and the size of the first logo corresponds to the first size; when the target user's upper body When the torso is not completely located in the capture area of the camera, and the target user is leaving the capture area, the user interface is controlled to no longer display the first logo.
  • Some embodiments of the present application provide a display device, including: a camera for capturing images; a display for displaying a user interface; an external input device for inputting a current video played on the user interface; a controller configured to The following steps are: obtaining a screenshot of the user interface when it is determined based on the captured image obtained by the camera that the user is changing channels; configuring optimized image parameters and optimized sound parameters for the current video played on the user interface according to the screenshot.
  • Some embodiments of the present application provide an application for human posture detection, including: determining, based on the acquired captured images, when a user changes channels, acquiring a screenshot of the user interface; configuring and optimizing image parameters for the current video played on the user interface according to the screenshot, and optimize sound parameters.
  • Some embodiments of the present application provide a display device, including: a display configured to display an image; a controller connected to the display, the controller configured to: perform a human gesture on a current frame image in a video to be detected Detect, determine the human body detection frame in the current frame image; determine the size and/or position of the target human body detection frame in the current frame image compared to the target human body detection frame in the previous frame image of the current frame image Whether there is a change in the frame; if the size and/or position of the target human detection frame in the current frame image has changed compared to the target human detection frame in the previous frame image, and the magnitude of the change is less than the preset change threshold, Then, based on the target human body detection frame in the previous frame image, adjust the size and/or position of the target human body detection frame in the current frame image, and send the adjusted current frame image to the display for processing. show.
  • Some embodiments of the present application provide a human body posture detection method, including: performing human body posture detection on a current frame image in a video to be detected, determining a human body detection frame in the current frame image; determining a target human body in the current frame image Whether there is a change in the size and/or position of the detection frame compared to the target human detection frame in the previous frame of the current frame image; if the size and/or position of the target human detection frame in the current frame image Compared with the target human body detection frame in the previous frame image, there is a change, and the change range is smaller than the preset change threshold, then based on the target human body detection frame in the previous frame image, the target in the current frame image is adjusted. The size and/or position of the human body detection frame, and the adjusted current frame image is displayed.
  • FIG. 1 is a schematic diagram of an operation scenario between a display device and a control apparatus according to one or more embodiments of the present application;
  • FIG. 2 is a block diagram of a hardware configuration of a display device 200 according to one or more embodiments of the present application;
  • FIG. 3 is a block diagram of the hardware configuration of the control device 100 according to one or more embodiments of the present application;
  • FIG. 4 is a schematic diagram of software configuration in a display device 200 according to one or more embodiments of the present application.
  • FIG. 5 is a schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present application;
  • FIG. 6 is a schematic diagram of portrait tracking according to one or more embodiments of the present application.
  • 7-8 are schematic diagrams of AI fitness scenarios according to one or more embodiments of the present application.
  • FIG. 10 is a schematic diagram of an application program interface of a display device according to one or more embodiments of the present application.
  • FIG. 11 is a schematic diagram of a display device dodgeball game user interface according to one or more embodiments of the present application.
  • FIG. 12 is a schematic diagram of a display device dodgeball game user interface according to one or more embodiments of the present application.
  • FIG. 13-14 are schematic diagrams of a display device identifying a user's limb according to one or more embodiments of the present application.
  • 15A-15B are schematic diagrams of a display device recognizing a user's channel change action according to one or more embodiments of the present application.
  • 16-18 are schematic diagrams of display interfaces of a display device according to one or more embodiments of the present application.
  • FIG. 1 is a schematic diagram of an operation scenario between a display device and a control device according to one or more embodiments of the present application.
  • a user can operate the display device 200 through a mobile terminal 300 and the control device 100 .
  • the control apparatus 100 may be a remote control, and the communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and wireless or other wired ways to control the display device 200 .
  • the user can control the display device 200 by inputting user instructions through keys on the remote control, voice input, control panel input, and the like.
  • mobile terminals, tablet computers, computers, notebook computers, and other smart devices may also be used to control the display device 200 .
  • the mobile terminal 300 may install a software application with the display device 200 to implement connection communication through a network communication protocol, so as to achieve the purpose of one-to-one control operation and data communication.
  • the audio and video content displayed on the mobile terminal 300 may also be transmitted to the display device 200 to realize a synchronous display function.
  • the display device 200 also performs data communication with the server 400 through various communication methods.
  • the display device 200 may be allowed to communicate via local area network (LAN), wireless local area network (WLAN), and other networks.
  • the server 400 may provide various contents and interactions to the display device 200 .
  • the display device 200 may be a liquid crystal display, an OLED display, or a projection display device.
  • the display device 200 may additionally provide an intelligent network television function that provides a computer-supported function in addition to the function of broadcasting and receiving television.
  • FIG. 2 exemplarily shows a configuration block diagram of the control apparatus 100 according to an exemplary embodiment.
  • the control device 100 includes a controller 110 , a communication interface 130 , a user input/output interface 140 , a memory, and a power supply.
  • the control device 100 can receive the user's input operation instruction, and convert the operation instruction into an instruction that the display device 200 can recognize and respond to, and play an intermediary role between the user and the display device 200 .
  • the communication interface 130 is used for external communication, and includes at least one of a WIFI chip, a Bluetooth module, NFC or an alternative module.
  • the user input/output interface 140 includes at least one of a microphone, a touchpad, a sensor, a key or an alternative module.
  • FIG. 3 is a block diagram showing a hardware configuration of the display apparatus 200 according to an exemplary embodiment.
  • the display device 200 includes a tuner 210 , a communicator 220 , a detector 230 , an external device interface 240 , a controller 250 , a display 260 , an audio output interface 270 , a memory, a power supply, and a user interface 280 .
  • the controller includes a central processing unit, a video processing unit, an audio processing unit, a graphics processing unit, a RAM, a ROM, and a first interface to an nth interface for input/output.
  • the display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen.
  • the tuner-demodulator 210 receives broadcast television signals through wired or wireless reception, and demodulates audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.
  • the detector 230 is used to collect external environment or external interaction signals.
  • the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.
  • the controller 250 controls the operation of the display device and responds to user operations.
  • the controller 250 controls the overall operation of the display apparatus 200 .
  • a user may input a user command on a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface (GUI).
  • GUI graphical user interface
  • the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.
  • a "user interface” is a medium interface for interaction and information exchange between an application program or an operating system and a user, which enables conversion between an internal form of information and a form acceptable to the user.
  • the commonly used form of user interface is Graphical User Interface (GUI), which refers to a user interface related to computer operations displayed in a graphical manner. It can be an icon, window, control and other interface elements displayed on the display screen of the electronic device, wherein the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, Widgets, etc. at least one of the visual interface elements.
  • GUI Graphical User Interface
  • FIG. 4 is a schematic diagram of software configuration in the display device 200 according to one or more embodiments of the present application.
  • the system is divided into four layers. Layer)
  • the Application Framework layer referred to as the "framework layer”
  • the Android runtime (Android runtime)
  • the system library layer referred to as the “system runtime layer”
  • the kernel layer contains at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power supply drive etc.
  • FIG. 5 is a schematic diagram showing an icon control interface of an application in a display device 200 according to one or more embodiments of the present application.
  • the application layer includes at least one application that can display a corresponding icon control on the display. , such as: live TV application icon control, video on demand application icon control, media center application icon control, application center icon control, game application icon control, etc.
  • Live TV app that can provide live TV from different sources.
  • Video-on-demand application that can provide video from different storage sources. Unlike live TV applications, video-on-demand provides a display of video from certain storage sources.
  • a media center application that can provide a variety of multimedia content playback applications. The application center can provide storage of various applications.
  • FIG. 6 is a schematic diagram of portrait tracking according to one or more embodiments of the present application.
  • the camera 231 as a detector 230 may be built in or externally connected to the display device 200 , and after starting the operation , the camera 231 can detect image data.
  • the camera 231 can be connected to the controller 250 through an interface component, so as to send the detected image data to the controller 250 for processing.
  • the camera 231 may include a lens assembly and a pan/tilt assembly, the lens assembly is disposed on the pan/tilt assembly, and the pan/tilt assembly may drive the lens assembly to rotate, so as to change the orientation of the lens assembly.
  • the lens assembly can capture video of users at different positions, so as to obtain user image data. Obviously, different orientations correspond to image capture in different areas.
  • the first rotating shaft on the pan/tilt assembly can drive the fixing piece and the lens assembly to rotate to the left, so that the In the captured image, the position of the user's portrait is located in the central area of the screen; when the imaging position of the user's body is lower, the lens assembly can be rotated upward through the second rotating shaft in the gimbal assembly to raise the shooting angle and make the user's portrait position located in the center area of the screen.
  • an automatic control program can be set in the display device 200, so as to adjust the orientation of the lens assembly in the camera 231 by detecting the position of the user, and repeat the process of detecting the position of the portrait according to a certain frequency, so as to realize the tracking of the position of the portrait , which includes the following steps:
  • Detect portrait position After the camera 231 starts to run, multiple frames of images are captured in real time, and the captured images are sent to the controller 250 of the display device 200 .
  • the controller 250 can, on the one hand, perform image processing according to the activated application program, for example, control the display 275 to display the image;
  • the detection of the portrait position can be completed by an image processing program. That is, body information is detected by capturing images captured by the camera 231 in real time.
  • the limb information can include key points and an outer frame that wraps the limb, and the position of the portrait is determined by the position information of the detected key points and the position of the limb frame in the image.
  • the key points can refer to a series of points in the human body image that can represent the characteristics of the human body. For example, eyes, ears, nose, neck, shoulders, elbows, wrists, waist, knees, ankles, etc.
  • the determination of key points can be obtained through image recognition, that is, the image corresponding to the key points can be determined by analyzing the characteristic shape in the picture and matching with the preset template, and the position corresponding to the image can be obtained, so as to obtain the position corresponding to each key point. .
  • the position can be represented by the number of pixels in the image from the boundary. According to the resolution and viewing angle of the camera 231, a plane rectangular coordinate system can be constructed with the upper left corner of the image as the origin and the right and downward directions as the positive directions, then each pixel in the image can pass through this rectangular coordinate system. to express.
  • the positions of some key points will change.
  • the relative position of the human body in the image captured by the camera 231 will also change. For example, when the human body moves to the left, the position of the human body in the image captured by the camera 231 will be shifted to the left, which is inconvenient for image analysis processing and real-time display.
  • FIGS 7 to 8 are schematic diagrams of AI fitness scenarios according to one or more embodiments of the present application, but in practical applications, due to the influence of the user's posture and different requirements in different applications, the center is used in some application scenarios. Position as a way of judging the position of a portrait cannot obtain better display, detection and tracking effects. In order to obtain a more accurate portrait position judgment, in some embodiments, taking an AI fitness scene as an example, as shown in Figures 7 and 8, after identifying multiple key points, a schematic diagram of a skeletal line can also be established according to the identified key points shape, so as to further determine the position of the portrait according to the skeletal line graphics. Among them, the skeleton line can be determined by connecting multiple key points. Under different user poses, the shape of the skeleton line is also different.
  • the drawn skeletal line can also dynamically adjust the shooting position of the camera according to the movement change rule of the skeletal line. For example, when judging that the movement state of the skeletal line changes from a squatting state to a standing state, the camera's angle of view can be raised, so that the portrait in the standing state can also be in a suitable area in the image, that is, from Figure 7 Transition to the effect shown in Figure 8. When judging that the change process of the motion state of the skeletal line is from the standing state to the squatting state, this can reduce the viewing angle of the camera, so that the portrait in the squatting state can also be in the appropriate area in the image, that is, the transition from Figure 8 to Figure 8 7 shows the effect.
  • the AI fitness scene in the fitness follow-up mode, a locked person needs to be tracked, and actions are monitored, follow-up data is generated, and follow-up results are counted.
  • the mobile phone scene if the face or body is close to the screen, and the face or body occupies a large proportion of the screen, the relative movement distance of the detected character image in each frame of image is small, and the phenomenon of loss (out of picture) will generally not occur. .
  • the display device is different from the mobile phone scene. When a person interacts with the display device, the distance between the person and the display device is generally far, the screen ratio of the face or body is small, and the moving distance between the front and rear frame images will be large.
  • a person is on the screen. Moving quickly in front of you can easily lead to the loss of character recognition, that is, the phenomenon of painting. Since the current fitness functions are mostly based on body information for body tracking, the camera to realize the portrait following function usually uses the recognition of faces or limbs as the identification basis. Regardless of body or face information, an ID will be generated each time it is identified as the identification of the identified body or face. However, when the identification is lost and then re-identified, that is, the character is drawn and re-entered, a new ID information will be regenerated, resulting in inconsistent IDs before and after the same person. In this way, when a specific person is tracked by a fitness or camera, the loss will cause irreversible, and also It is impossible to track the effectiveness of the same person.
  • the controller of the display device is configured to perform the following steps:
  • the person image information collected by the camera includes image information of at least one person, and the person's body movements and facial feature information, etc. can be read from the image information.
  • the display device cannot determine which character's instruction is used as the control instruction to respond, that is, it cannot determine which character is used as a specific character for tracking. Therefore, it is necessary to determine the locked and tracked character during interaction.
  • the locked and tracked character is one of the characters that interacts with the display device, and the subsequent interaction only responds to the instructions generated by the locked and tracked character. In order to achieve locked tracking of the same person, it is necessary to select one of the characters as the locked tracking character when multiple characters interact with the display device.
  • the locked and tracked person is that person.
  • the judgment can be made according to whether each character has made a specific action, and the action recognition of the character can be determined according to the key point information of the body of the character.
  • S13 Establish an association relationship between the specific body ID information of the locked and tracked person and the facial feature information, and obtain the facial feature value of the locked and tracked person.
  • each character has its own facial feature information, and different characters have different facial feature information. Therefore, each person can be identified by the facial feature information, and if the same or similar facial feature information is identified, it can be identified as the same person.
  • the corresponding specific limb ID information is lost, that is, the corresponding specific limb ID information is deleted after the person exits the picture. If the character enters the screen again, the corresponding body ID information will be regenerated, and it is easy to identify the same person as two people.
  • an association relationship is established between the specific body ID information and the facial feature information of the locked and tracked person, and the specific body ID information and the facial feature information associated with each other are used as the facial feature value of the locked and tracked person, so that the locked and tracked person is Tracking characters for identification, facial feature information will not be deleted when the characters are drawn, and will always be saved in the controller.
  • the controller is further configured to perform the following steps when performing locked and tracked characters based on facial feature values:
  • Step 1311 When the camera application is an AI fitness application, determine that the locked and tracked person is a fitness person.
  • Step 1312 based on the facial feature values of the fitness personnel, continuously collect follow-up exercise information of the fitness personnel for follow-up exercises based on the demonstration video presented in the user interface.
  • Step 1313 Generate a follow-up exercise picture based on the follow-up exercise action information, display it on the user interface, and display the follow-up exercise picture on the side of the picture where the demonstration video is located.
  • the AI fitness function configured by the display device can be used for one or more characters. If there is one person, the fitness person is the locked and tracked person. Demonstration videos are presented in the user interface of the monitor for easy follow-up by fitness personnel.
  • the camera application is an AI fitness application, and the AI fitness application calls the camera to always capture the follow-up movements of the locked and tracked characters, and display them in the user interface on the display.
  • the display device can also apply human posture detection to game applications.
  • the following will take the display device's dodgeball display technology scheme based on body recognition to predict the movement trend, and the user interface as an example.
  • FIG. 9 is an application UI interface according to one or more embodiments of the present application.
  • the application UI interface includes 4 applications that have been installed on the TV, namely news headlines, theater on-demand , AR dodgeball, K song, etc.
  • the remote control, voice, etc. By using the remote control, voice, etc. to move the focus on the display, you can select different applications or other function buttons.
  • the TV display screen is configured to display other interactive elements while displaying the application UI interface
  • the interactive elements may include, for example, TV homepage controls, search controls, message button controls, mailbox controls, browsing Controls, Favorites Controls, Signal Bar Controls, etc.
  • the controller of the display device in the embodiments of the present application controls the UI of the TV in response to the operation of the interactive element. For example, when a user clicks a search control through a controller such as a remote control, the search UI can be displayed on top of other UIs, that is, the UI of the application component that controls the mapping of interactive elements can be enlarged, or run and displayed in full screen.
  • the interactive element may also be operable by a sensor, which may be, but is not limited to, an acoustic input sensor, such as a microphone, that can detect voice commands that include an indication of the desired interactive element.
  • a sensor which may be, but is not limited to, an acoustic input sensor, such as a microphone, that can detect voice commands that include an indication of the desired interactive element.
  • the user may identify the desired interactive element using a "dodgeball" or any other suitable identification, such as a search control, and may also describe the desired action to be performed in relation to the desired interactive element.
  • the controller may recognize voice commands and submit data characterizing the interaction to the UI or its processing components or engines.
  • FIG. 10 is a schematic diagram of an application program interface of a display device according to one or more embodiments of the present application.
  • the user can control the focus of the display screen through the remote control, select the AR dodgeball application, so that its icon is highlighted in the user interface of the display screen; then click the highlighted icon to open the application Icon mapping application.
  • FIG. 11 is a schematic diagram of a display device dodgeball game user interface according to one or more embodiments of the present application.
  • the display device provided by the present application includes a camera, a display, and a controller.
  • the camera is used to collect the target user and background image in the dodgeball game in the detection area;
  • the display is used to display the dodgeball game user interface including the target user and the background image;
  • the controller is configured to: according to the background image The first size of the target user in the image, and the first position of the target user in the user interface, the user interface is controlled to display a first mark at the first position, where the first mark is used for Trigger the launch of the dodge ball, and the size of the first sign corresponds to the first size, as shown in FIG. 11 .
  • the controller controls the user interface to no longer display the first logo .
  • an image of a target user and a background image of the target user are acquired in the camera restart monitoring area, wherein both the target user and the background image will be displayed on the user interface of the dodgeball game.
  • the controller controls the user interface to display the first logo at the first position according to the first size of the target user in the background image and the first position of the target user in the game user interface.
  • the controller determines the size of the first logo according to the size of the target user in the background image, that is, the user who is playing the dodgeball game, that is, the larger the size of the user in the background image, the larger the first logo; The smaller the size is, the smaller the first identifier is.
  • the controller When the user moves from the first position displayed in the user interface to the second position, the controller will re-determine the size of the second logo according to the second size of the user at the second position, and display the second logo on the game user interface .
  • the controller controls the user interface to display a first logo at the first position, and the size of the first logo corresponds to the first size, which specifically includes: when the target user is relative to the When the camera moves from far to near, the first size gradually increases, and the size of the first sign increases corresponding to the first sign; when the target user moves from near to far relative to the camera When , the first size gradually becomes smaller, and the size of the first marker decreases correspondingly to the size of the first marker.
  • the controller can correspondingly control the second logo to be enlarged compared to the first logo, or Zoom out.
  • the controller controls the first identification to cover the target user displayed at the first location.
  • the first logo can be displayed as an approximate rectangular frame, and when the controller locates the first position of the target user in the game user interface, the approximate rectangular frame that conforms to the current first size of the target user, that is, the first logo is displayed overlaid to target users.
  • FIG. 12 is a schematic diagram of a display device dodgeball game user interface according to one or more embodiments of the present application. Referring to FIG. 12 , when the upper torso of the target user is not completely located in the collection area, and the target user is leaving the collection area, the controller controls the user interface to no longer display the first logo.
  • the target user when the target user leaves the game, it includes: the target user walks towards the camera capture area but the user's upper torso is still in the capture area, the target user is passing through the edge of the camera capture area and part of the user's upper torso is still in the capture area, The target user has passed the edge of the camera capture area and the user's upper torso is completely outside the capture area.
  • the display device provided by this application can recognize that the target user is passing through the edge of the camera capture area and the user's upper torso is still in the capture area, and controls the user interface to no longer have the first mark, and will not trigger the game to continue to launch and avoid
  • the ball as shown in the figure, is an approximate rectangular box with a dotted line, which is used to facilitate understanding and is not displayed on the game user interface of the display device.
  • the approximate rectangular box with a dotted line represents part of the target recognized by the controller User upper torso.
  • FIGS. 13-14 are schematic diagrams of a display device identifying a user's limb according to one or more embodiments of the present application.
  • the controller controls the user interface to display a first mark at the first position, and the size of the first mark corresponds to the first size, and specifically includes the controller: the camera captures the target in the image The user's left arm elbow is identified as the first positioning point, and the right arm elbow is identified as the second positioning point; according to the first separation distance between the first positioning point and the second positioning point, it is determined that the The size of the first logo is displayed at the location.
  • the controller decomposes the upper body torso of the target user collected by the camera into various points according to the image recognition model, judges the movement trend of the user according to the position changes of each point of the upper limb, and infers the overall position from some limbs, thereby changing the position and size of the first mark, Its position is the first position provided by this application, and its size is the size of the first logo provided by this application.
  • the controller identifies the left arm elbow of the target user as the first anchor point and the right arm elbow as the second anchor point, and then according to the first separation distance between the first anchor point and the second anchor point
  • the first separation distance is 10cm
  • the controller controls the user interface to display a first mark with a width of 10cm at the first position, and the first mark may be implemented as an approximate rectangular frame.
  • the controller identifies the target user in the image captured by the camera through the following steps. First, extract a single-frame image from the captured video every preset number of frames; perform human body detection on the single-frame image to determine whether the single-frame image contains a human body; if the single-frame image contains a human body Human body, perform face detection on the single-frame image within the human body frame range of the human body, and determine whether the human body frame range contains a human face; if the human body frame range includes a human face, the human face Perform feature extraction to obtain the facial features of the user in the collected video; compare the facial features of the user in the collected video with a preset database of family members' facial features, and determine the features in the collected video. Whether the user is a family member and a game user.
  • the controller applies a face detection algorithm (for example: Deformable Part Model) to the human body in the image captured by the camera, and performs face detection on the single-frame image within the detected human frame. Detect to determine whether there is a face within the body frame. If a human face is included in the range of the human body frame, feature extraction is performed on the face to obtain the face feature of the user. The facial features of the user are compared with the facial feature database of family members to determine whether the user is a family member and a game user.
  • a face detection algorithm for example: Deformable Part Model
  • the controller is further configured to: identify the left arm elbow of the target user in the image captured by the camera as the first anchor point, the right arm elbow as the second anchor point, and other joints to identify is the third anchor point, the third anchor point is located at the left hand, and/or the left shoulder, and/or the neck, and/or the right shoulder, and/or the right hand, and/or the left waist, and/or the right waist; according to the tracking The position changes of the first positioning point, the second positioning point, and the third positioning point determine whether the upper torso of the target user is completely in the collection area, and whether the target user is leaving the collection area.
  • the user's left hand is identified as point 1
  • the left arm elbow is identified as point 2
  • the left shoulder is identified as point 3
  • the neck is identified as point 4
  • the right shoulder is identified as point 5
  • the right arm elbow is identified as point 6
  • the right hand is identified as point 6.
  • the first positioning point may be implemented as point 2, the second positioning point may be implemented as point 6, and the third positioning point may be implemented as one or a combination of the remaining points.
  • the controller identifies the above-mentioned multiple points in the collected image, tracks the multiple points, and calculates the overall position of the user according to the multiple points, for example, according to 9 points, to obtain the first position; wherein, between point 2 and point 6 The distance between them determines the size of the first identification display frame, the movement direction of the target user is determined by the position changes of the multiple points, and it is determined whether the upper body torso of the target user is completely in the collection area, and whether the target user is Leave the collection area.
  • the controller extracts a single frame image from the captured video every preset number of frames (eg, every 90 frames) for the captured image received from the camera.
  • Apply the human body detection algorithm in image recognition technology for example: the convolutional pose machine Convolutional Pose Machine, detect each joint point of the human body, and then determine the range of these joint points as the human body frame of the human body
  • the single frame image Perform human body detection, determine the movement direction of the target user through the positional changes of multiple points, determine whether the upper body torso of the target user is completely in the collection area, and whether the target user is leaving the collection area.
  • the family member feature database stores the physical features of each family member and the corresponding family member identifiers.
  • the body feature of the family member stores the coordinates of each body feature point of the family member. For example: the coordinates of the right hand of family member 03 are (10, 0), the coordinates of the left hand are (-10, 0), the coordinates of the right shoulder are (5, -10), the coordinates of the left shoulder are (-5, -10), etc. .
  • connection rules for example: connect the left shoulder to the right shoulder, then connect the right shoulder to the elbow of the right arm, and then connect the elbow of the right arm to the right hand
  • the feature points of the upper body of the user are sequentially connected to obtain the upper body edge map of the user. In the same way, the upper body edge map of each family member is obtained.
  • the degree of similarity between each connection line of the user's upper body edge map and the corresponding connection line of the family member's upper body edge map is determined by the angle of the connection line.
  • the angle of the line connecting the left hand, the left elbow, and the left shoulder in the upper body edge diagram of the user is 27 degrees.
  • the angle of the line connecting the left hand, the left elbow, and the left shoulder in the upper body edge image of the target user is 30 degrees.
  • the similarity between the connection line of the upper body edge graph of the target user and the connection line of the upper body edge graph of family member A is 1-(
  • /30) 0.9.
  • the similarity between each connection line of the target user's upper body edge map and the corresponding connection line of the family member's upper body edge map is similar.
  • the mean value of the degree is determined as the similarity between the upper body characteristics of the user and the upper body characteristics of the family member.
  • the present application also provides a dodgeball display method based on body recognition, the method comprising: according to the background image of the user interface of the dodgeball game The first size of the target user and the first position of the target user in the user interface, and the user interface is controlled to display a first mark at the first position, where the first mark is used to trigger avoidance The ball is launched, and the size of the first identification corresponds to the first size; when the upper torso of the target user is not completely located in the capture area of the camera, and the target user is leaving the capture area, control the The user interface no longer displays the first logo.
  • the specific operations and steps of the body recognition-based dodgeball display method have been described in detail in the implementation scheme of the display device above, and will not be repeated here.
  • controlling the user interface to display a first logo at the first position, where the size of the first logo corresponds to the first size specifically includes: when the target user moves from far to near relative to the camera When moving, the first size gradually increases, and the size of the first logo increases correspondingly to the first logo; when the target user moves from near to far relative to the camera, the first size gradually decreases , the size of the first mark decreases corresponding to the size of the first mark.
  • controlling the user interface to display a first logo at the first position, where the size of the first logo corresponds to the first size specifically includes: placing the left elbow of the target user in the captured image The part is identified as the first positioning point, and the elbow of the right arm is identified as the second positioning point; according to the first separation distance between the first positioning point and the second positioning point, it is determined to display the first positioning point at the first position.
  • the size of the logo The specific operations and steps of the body recognition-based dodgeball display method have been described in detail in the implementation scheme of the display device above, and will not be repeated here.
  • the method further includes: identifying the left arm elbow of the target user in the captured image as the first anchor point, the right arm elbow as the second anchor point, and the other joints as the third anchor point,
  • the third anchor point is located at the left hand, and/or the left shoulder, and/or the neck, and/or the right shoulder, and/or the right hand, and/or the left waist, and/or the right waist; according to tracking the first anchor point , the position change of the second positioning point and the third positioning point, to determine whether the upper body torso of the target user is completely in the collection area, and whether the target user is leaving the collection area.
  • controlling the user interface to display the first logo at the first position specifically includes the controller: covering the target user displayed at the first position with the first logo.
  • the beneficial effects of the embodiments of the present application are that, by constructing the first size, the size of the first sign can be controlled; further, by constructing the first position, the location of the first sign can be acquired; and by controlling the first sign not to be displayed, it can be realized
  • the user does not launch the dodge ball when leaving the game; by constructing the first positioning point, the second positioning point, and the third positioning point, it is possible to identify the body of the game user, predict the movement trend of the game user, and adjust the first positioning point according to the size displayed by the user on the screen.
  • the size of the logo so that the user does not play the game when the user is not within the screen display range.
  • the controller determines that the user is changing channels based on the captured image obtained by the camera, which specifically includes:
  • the controller analyzes the captured images obtained by the camera at the first frequency to identify user presence. Specifically, after the display device is powered on, the controller controls the camera to capture images and obtain an image preview thereof, and detects whether a user is watching the display device by using an image recognition algorithm on the preview image. Before the controller detects the user, the controller analyzes the captured image at the first frequency. For example, the controller performs image recognition analysis on the image captured by the camera at intervals of 10 frames to detect whether there is a user in the detection range of the camera. The setting of the first frequency can effectively save the computing resources of the controller.
  • the controller After recognizing the user, the controller analyzes the captured image at the second frequency to recognize the upper limb movement of the user, and when it is recognized that the user's elbow and hand position change, and the height difference between the hand position and the elbow position is less than the height threshold, It is recognized that the user is performing a channel change action, and the second frequency is greater than the first frequency.
  • 15A-15B are schematic diagrams of a display device recognizing a user's channel-changing action according to one or more embodiments of the present application.
  • the controller after the camera acquires a captured image about the user, the controller sends the captured data frame to The image recognition model analyzes to determine whether the user has performed a channel change action.
  • the image recognition model identifies the user's upper limbs, ie, arms and torso with lines and nodes. For example, the elbow is identified as point 1 and the hand is identified as point 2. Before the user changes channels, their relative positions are shown in Figure 15A.
  • the controller analyzes the The camera collects the data frame in the image and uses the image recognition model to identify it.
  • the controller determines and recognizes that the user's action has a high possibility of picking up the remote control for the user.
  • the controller switches the first film source to the second film source, that is, the channel change action is performed;
  • the trigger controller sends a first instruction including a screenshot of the second film source to the server, where the first instruction is used to make the server identify the second film source according to the screenshot to determine whether it can provide optimizing the image parameters and optimizing the sound parameters; when the server can provide the optimized image parameters and the optimized sound parameters, the controller controls the user interface to play the second film source configured by the optimized image parameters and the optimized sound parameters, As shown in Figure 15B.
  • the display device recognizes the user's limb to perform a channel-changing operation as follows:
  • the frequency of preview analysis is accelerated when the display device detects the user.
  • the display device controller determines that the user is changing channels based on the captured image obtained by the camera, and specifically includes the controller: after recognizing the user's channel-changing action based on the captured image, determining whether there is any audio output from the display device within the first time period. Instantaneous silence; if yes, it is determined that the user has switched channels; otherwise, it is determined that the user has not switched channels.
  • the controller analyzes the captured image at the first frequency to identify whether a user is watching the display device. For example, the controller performs image recognition and analysis on the images collected by the camera at intervals of 10 frames to detect whether a user exists in the detection range of the camera. By setting the first frequency, the computing resources of the controller can be effectively saved.
  • step 902 the display device performs user behavior analysis.
  • the controller analyzes the captured image at the second frequency to perform user behavior analysis, which may be implemented, for example, to recognize the movement of the user's upper limbs.
  • marking is made when the user's hand-raising motion is detected.
  • the controller identifies the changes in the position of the user's elbow and hand and marks it. For example, if the height difference between the hand position and the elbow position is less than the height threshold, the controller determines that the user has performed a suspected channel-changing operation.
  • step 904 the display device determines that the sound change is received and the interval time is less than 30 seconds.
  • the controller determines whether the audio output of the display device is momentarily muted within the first period of time since it is marked, that is, after identifying that the user has performed a suspected channel-changing operation.
  • the momentary silence usually occurs when the display device changes channels, Or when the film source is switched, the first time length can be implemented as 30 seconds, for example.
  • the controller determines that the audio output of the display device is momentarily muted within the first time length, that is, within the above 30 seconds, it is determined that the user switches the first film source to the second film source;
  • the first instruction of the screenshot of the second film source is sent to the server, and the first instruction is used to make the server identify the second film source according to the screenshot to determine whether to provide optimized image parameters and optimized sound parameters;
  • the controller controls the user interface to play the second film source configured by the optimized image parameters and the optimized sound parameters. It should be noted that the identification of the currently playing video according to the screenshot obtained after the channel change can also be implemented locally on the display device.
  • step 904-1 If the controller determines that no momentary silence occurs in the audio output of the display device within the first time period, that is, within the above 30 seconds, it is determined that the user has not switched the second film source, and the analyzed data is discarded, as shown in step 904-1.
  • controlling to configure optimized image parameters and optimized sound parameters for the current video played on the user interface according to the screenshots specifically includes the controller: sending a first instruction including the screenshot to the server, the first instruction for enabling the server to identify the current video according to the screenshot to determine available optimized image parameters and optimized sound parameters; receive a second instruction sent from the server including optimized image parameters and optimized sound parameters; The second instruction configures the optimized image parameters and optimized sound parameters for the current video played by the user interface. For example, when the controller of the display device recognizes and determines that the user picks up the remote control to switch the first video source played on the user interface to the second video source, it will take a screenshot of the second video source played on the current user interface; then the controller will The screenshot is sent to the server through the first instruction.
  • the screenshot is a screenshot of the logo of the display device at the corner position of the user interface of the current video, and/or a screenshot of the text of the current video at the edge position of the user interface.
  • the screenshot is a screenshot of the logo of the display device at the corner position of the user interface, and/or a screenshot of the text of the second image source at the edge position of the user interface.
  • the logo of the display device is usually located at the upper left corner of the screen of the display device.
  • the program name is usually located on the top, bottom, or side of the screen of the display device.
  • the controller can improve image recognition by taking a screenshot of the text information at the edge of the screen for the server to recognize. efficiency, reducing the amount of data sent, as shown in step 904-2.
  • the present application also provides a method for automatically configuring video parameters at the display device side, the method includes: determining, based on the acquired captured images, that when the user changes channels, acquiring the user A screenshot of the interface; according to the screenshot, the optimized image parameters and the optimized sound parameters are configured for the current video played on the user interface.
  • determining that the user is changing channels based on the acquired captured images specifically includes: after recognizing the user's channel-changing action based on the captured images, determining whether the audio output is momentarily muted within a first time period; if so, determining whether The user switches channels; otherwise, it is determined that the user does not switch channels.
  • determining that the user is changing channels based on the captured image specifically includes: analyzing the captured image at a first frequency to identify the presence of the user; after identifying the user, analyzing the captured image at a second frequency to identify the user's presence
  • analyzing the captured image at a second frequency to identify the user's presence
  • For upper limb movement when it is recognized that the user's elbow and hand position change, and the height difference between the hand position and the elbow position is less than the height threshold, it is recognized that the user is performing a channel change action, and the second frequency is greater than the first frequency.
  • configuring optimized image parameters and optimized sound parameters for the current video played on the user interface according to the screenshot specifically includes: sending a first instruction including the screenshot to the server, where the first instruction is used to make The server identifies the current video according to the screenshot to determine available optimized image parameters and optimized sound parameters; receives a second instruction sent from the server including optimized image parameters and optimized sound parameters; according to the second The instruction is to configure the optimized image parameters and optimized sound parameters for the current video played by the user interface.
  • the screenshot is a screenshot of the logo of the display device at the corner position of the user interface of the current video, and/or a screenshot of the text of the current video at the edge position of the user interface.
  • the present application also provides an automatic video parameter configuration for the server, the method comprising: receiving a first instruction sent by a display device and containing a screenshot of a second source. ; Identify the name of the second film source based on the screenshot included in the first instruction, to determine whether optimized image parameters and optimized sound parameters corresponding to the second film source can be provided; the optimized image can be provided When optimizing the parameters and sound parameters, send a second command including the optimized image parameters and the optimized sound parameters to the display device.
  • the specific operations and steps for the server to realize the automatic configuration of video parameters have been described in detail in the server implementation scheme above, and will not be repeated here.
  • the beneficial effects of the embodiments of the present application are that, by collecting the user image through the camera, real-time detection of the user's physical movement of changing channels can be realized; further, by obtaining a screenshot of the user interface, the currently playing video can be recognized; further, the sound image can be optimized by configuring the recognized video.
  • Parameters when the display device is connected to the set-top box through HDMI and used as a display, it can improve the accuracy of identifying the user's channel change, identify the program information, and automatically configure the playback parameters after the channel change in time.
  • FIGS. 16-18 are schematic diagrams of display interfaces of a display device according to one or more embodiments of the present application.
  • the left area is an image of the coach's action display
  • the right area is an image of the user's posture captured by the camera in real time.
  • any slight difference in the two frames of pictures before and after, such as light, frame rate, background, etc., will lead to differences in the detection results, and the human body detection frame will shake during the visual display process, which will affect the user experience. .
  • the embodiment of the present application provides a display device, which can prevent the human body detection frame in the human body posture detection result from shaking due to the slight difference in the two adjacent frames of pictures, thereby improving the visual display effect.
  • a display device which can prevent the human body detection frame in the human body posture detection result from shaking due to the slight difference in the two adjacent frames of pictures, thereby improving the visual display effect.
  • the display device 200 may collect the motion video of the user through the camera 201, and send the collected motion video as the video to be detected to the controller 250 for processing.
  • the controller 250 uses the human posture detection network model to perform human posture detection on each frame of the image in the video to be detected, and determines the information related to the person in each frame of the image, and the information includes the person The position information, including the center point position (x c , y c ), width (w c ), and height (h c ) of the human detection frame; and human body posture information, including the position information of human joint points (x loc , y loc ) .
  • the position of the center point (x c , y c ) of the human body detection frame can be used to determine the position of the human body detection frame, and the width (w c ) and height (h c ) of the human body detection frame can be used to determine the size of the human body detection frame .
  • the controller 250 determines the current frame based on the center point position of the target human body detection frame in the current frame image and the center point position of the target human body detection frame in the previous frame image. Whether there is a change in the position of the target human detection frame in the image compared to the target human detection frame in the previous frame image; and/or, based on the width and height of the target human detection frame in the current frame image, and the previous frame image The width and height of the target human body detection frame in the image determine whether the size of the target human body detection frame in the current frame image has changed compared with the target human body detection frame in the previous frame image.
  • the variation range of the above-mentioned changes is smaller than the preset change threshold, it can be considered that the above-mentioned changes are not caused by changes in the user's posture, but are caused by some slight differences between the two frames of pictures before and after.
  • the size and/or position of the target human detection frame in the current frame image can be adjusted based on the target human detection frame in the previous frame image, so that the size and/or the target human detection frame in the current frame image can be adjusted.
  • the position is the same as the target human detection frame in the previous frame image, and then the adjusted current frame image is sent to the display for display.
  • the detection of human body posture can be effectively avoided.
  • the human detection frame in the result jitters due to the slight difference between the two adjacent frames, which improves the visual display effect.
  • the magnitude of the change is greater than or equal to the preset change threshold, it can be considered that the change is caused by a change in the user's posture.
  • the current frame image can be directly sent to the display for display.
  • the user can observe that the human body detection frame displayed on the display screen will change with the change of the user's posture, so as to ensure the accuracy of the visual display of the human body posture detection result.
  • the display device provided by the embodiment of the present application can determine the target human body detection frame in the current frame image by comparing the size and/or position of the target human body detection frame in the current frame image and the target human body detection frame in the previous frame image Whether there is a change in the size and/or position of the target human body detection frame in the current frame image, when the size and/or position of the target human body detection frame in the current frame image changes, and the change range is less than the preset change threshold, then the target human body detection frame in the previous frame image is detected based on The frame adjusts the size and/or position of the target human detection frame in the current frame image, and sends the adjusted current frame image to the display for display, which can prevent the human detection frame in the human posture detection result from being caused by two adjacent front and rear frames. Small differences in frame images appear jittery, improving the visual display effect.
  • the target human detection frame when there are many people in the detection scene at the same time, and the number of people is more than the number of people required for the detection scene (for example, when a single person is exercising and only one person needs to be detected) , the target human detection frame will switch back and forth between multiple people, showing the phenomenon of human detection frame drift.
  • the excess human body detection frame strategy may be adopted in the embodiment of the present application, and a human body detection frame with more than the number of people required is established during the human body posture detection process.
  • the current frame image includes two or more human body detection frames
  • determine the IOU Intersection-over-Union, Intersection-over-Union, Intersection-over-Union, Intersection-over-Union, Cross-union ratio
  • the center point position (x c , y c ), width and height information (w c , h c ) of each human detection frame in the current frame image, and the information in the previous frame image can be used.
  • the center point position (x c , y c ) and width and height information (w c , h c ) of the target human detection frame are used to calculate the respective human detection frames in the current frame image and the target human detection frame in the previous frame image. of IOUs.
  • the frame rate factor mainly affects the display accuracy of the target human body detection frame. Therefore, in this embodiment of the present application, an IOU threshold related to the frame rate can be set to reduce the frame rate detection of the target human body. The effect of the box can thus be adapted to different image capture devices and display devices.
  • the IOU threshold value can be calculated by formula 1,
  • k is an adjustable constant
  • ⁇ is the frame rate of the video to be detected.
  • the target human body detection frame in the current frame image after determining the target human body detection frame in the current frame image, determine whether the intersection ratio IOU of the target human body detection frame in the current frame image and the target human body detection frame in the previous frame image is greater than a preset IOU threshold; if the IOU of the target human detection frame in the current frame image and the target human detection frame in the previous frame image is greater than the preset IOU threshold, it is determined that the size and/or position of the target human detection frame in the current frame image are similar.
  • a change threshold may be set to determine whether there is jitter in the target human body detection frame.
  • the target human detection frame in the current frame image when judging whether the size and/or position of the target human detection frame in the current frame image is shaken compared to the target human detection frame in the previous frame image, if the size and/or position of the target human detection frame in the current frame image and / or the position is changed compared to the target human detection frame in the previous frame image, and the magnitude of the change is less than the preset change threshold, it is determined that the target human detection frame is shaking, at this time, based on the target human detection frame in the previous frame image , adjust the size and/or position of the target human body detection frame in the current frame image, and send the adjusted current frame image to the display for display.
  • the size and/or position of the target human detection frame in the current frame image does not change compared to the target human detection frame in the previous frame image, or the size and/or position of the target human detection frame in the current frame image is different If there is a change in the target human body detection frame compared to the previous frame image, and the change range is greater than or equal to the above preset change threshold, it is determined that the target human body detection frame does not shake, and the current frame image is sent to the display for display.
  • the magnitude of change ⁇ of the size and/or position of the target human detection frame in the current frame image compared to the target human detection frame in the previous frame image can be determined by formula 2:
  • It represents the size and/or position information of the target human body detection frame in the previous frame image
  • It -1 represents the size and/or position information of the target human body detection frame in the previous frame image
  • a change threshold related to the frame rate can be set to determine whether the target human detection frame shakes.
  • the above-mentioned preset change threshold is set according to formula 3;
  • ⁇ is an adjustable constant
  • ⁇ is the frame rate of the video to be detected.
  • the solid line frame in FIG. 17 is the target human body detection frame in the current frame image
  • the dotted line frame is the target human body detection frame in the previous frame image.
  • the size and/or position of the target human body detection frame has changed compared to the target human body detection frame in the previous frame of image. Therefore, when the above-mentioned variation is less than the preset change threshold, the intuitive feeling fed back to the user is the one in the display interface. There is jitter in the target human detection frame.
  • the size and/or position of the target human body detection frame in the current frame image changes compared to the target human body detection frame in the previous frame image, and the change range is smaller than the preset change Threshold, then based on the target human detection frame in the previous frame image, adjust the size and/or position of the target human detection frame in the current frame image, so that the size and/or position of the target human detection frame in the current frame image is the same as the previous frame.
  • the target human detection frame in one frame of image is the same.
  • the intuitive feeling fed back to the user is the target human body in the display interface.
  • the detection frame remains stationary.
  • the display device provided by the embodiment of the present application sets the above IOU threshold and the above variation threshold based on the frame rate of the video to be detected.
  • the influence of the frame rate on the judgment result can be effectively reduced.
  • the human body detection frame is prevented from shaking, and the visual display accuracy of the human body detection frame can also be guaranteed.
  • the floating point method in order to prevent the problem of joint point jitter during the visual display process, can be used to calibrate the position coordinates of the human joint point, and based on the illumination parameters, the stability of the display of the human joint point can be improved.
  • the human body joint point position (x loc , y loc ) in the current frame image can be converted into a floating-point human body joint point position (x f , y f ) by the following formula, as shown in the formula 4:
  • i 0, 1 , . ); ⁇ (0,1), representing the lighting parameter.
  • ⁇ is different under different lighting conditions.
  • v i, j is larger as a whole, and ⁇ should be set larger at this time, and when the light is insufficient or too strong, ⁇ should be set smaller, such as 0.05 .
  • the human body joints to be displayed are marked in the current frame image point, and send the current frame image marked with the joint points of the human body to be displayed to the display for display.
  • the display device provided by the embodiment of the present application performs floating-point processing on the human body joint point information in the current frame image based on the illumination parameters, which can effectively prevent the human body joint point from shaking when the human body posture detection structure is visually displayed. Improve the visual display effect,
  • Some embodiments of the present application also provide a method for detecting human body posture, the method comprising:
  • S602. Determine whether there is a change in the size and/or position of the target human body detection frame in the current frame image compared to the target human body detection frame in the previous frame image of the current frame image.
  • the size and/or position of the target human body detection frame in the current frame image is changed compared to the target human body detection frame in the previous frame image, and the change amplitude is less than a preset change threshold, then based on the previous frame image
  • the target human body detection frame is adjusted, the size and/or position of the target human body detection frame in the current frame image is adjusted, and the adjusted current frame image is displayed.
  • the human body posture detection method can determine the target human body in the current frame image by comparing the size and/or position of the target human body detection frame in the current frame image and the target human body detection frame in the previous frame image. Whether there is a change in the size and/or position of the detection frame, when the size and/or position of the target human detection frame in the current frame image changes, and the magnitude of the change is less than the preset change threshold.
  • the human body detection frame adjusts the size and/or position of the target human detection frame in the current frame image, and displays the adjusted current frame image, so as to prevent the human body detection frame in the human posture detection result from being caused by two adjacent frames. Small differences in the picture appear jittery, improving the visual display effect.
  • each human body detection frame in the current frame image and the target human body in the previous frame image are determined respectively.
  • the IOU of the detection frame; the human detection frame with the largest IOU in the current frame image and the target human detection frame in the previous frame image is determined as the target human detection frame in the current frame image.
  • the current frame image includes two human body detection frames, which are human body detection frame 1 and human body detection frame 2 respectively, then first determine the difference between human body detection frame 1 and the target human body detection frame in the previous frame image IOU1, and IOU2 of the human body detection frame 2 and the target human body detection frame in the previous frame image, if IOU1>IOU2, the human body detection frame 1 is used as the target human body detection frame in the current frame image; if IOU2>IOU1, then the The human body detection frame 2 is used as the target human body detection frame in the current frame image.
  • the intersection ratio IOU of the target human body detection frame in the current frame image and the target human body detection frame in the previous frame image is greater than the preset IOU threshold; if the target human body detection frame in the current frame image and the previous frame image If the IOU of the target human body detection frame in the image is greater than the preset IOU threshold, it is determined that the human body in the current frame image is not moved out; if the IOU of the target human body detection frame in the current frame image and the target human body detection frame in the previous frame image is less than or equal to With the above preset IOU threshold, it is determined that the human body in the current frame image has been moved out.
  • the above-mentioned preset IOU threshold is set according to formula 5,
  • k is an adjustable constant
  • ⁇ is the frame rate of the video to be detected.
  • prompt information is output, and the prompt information is used to prompt the user that the human body in the current frame image has been removed; if it is determined that the human body in the current frame image has not been removed, continue judging Whether the target human detection frame in the current frame image has changed.
  • the detection frame information of the target human body detection frame in the current frame image and the detection frame information of the target human body detection frame in the previous frame image are respectively determined, and the detection frame information includes the detection frame information of the human body detection frame. Center point coordinate information, width information and length information.
  • the current frame is determined according to the above difference value.
  • the magnitude of change in the size and/or position of the target human detection frame in the image If the variation is smaller than the preset variation threshold, adjust the size and/or position of the target human detection frame in the current frame image based on the target human detection frame in the previous frame image, and send the adjusted current frame image to Display for display; otherwise, send the current frame image to display for display.
  • the current frame image is sent to the display for display.
  • the above-mentioned preset change threshold is set according to formula 6
  • ⁇ is an adjustable constant
  • ⁇ is the frame rate of the video to be detected.
  • the human body posture detection method sets the IOU threshold and the change threshold based on the frame rate of the video to be detected, so that when judging whether the human body detection frame shakes, the influence of the frame rate on the judgment result can be effectively reduced , which can not only prevent the human body detection frame from shaking, but also ensure the visual display accuracy of the human body detection frame.
  • the following formula 7 can be used to convert the human body joint point positions (x loc , y loc ) in the current frame image to floating point Type body joint point position (x f , y f ):
  • i 0, 1 , . ); ⁇ (0,1), representing the lighting parameter.
  • ⁇ is different under different lighting conditions.
  • v i,j is larger as a whole, and ⁇ should be set larger at this time, and when the illumination is insufficient or too strong, ⁇ should be set smaller.
  • the human body posture detection method performs floating-point processing on the human body joint point information in the current frame image based on the illumination parameters, which can effectively prevent the human body joint points from shaking when the human body posture detection structure is visually displayed. problem, improve the visual display effect.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Controls And Circuits For Display Device (AREA)

Abstract

本申请一些实施方式公开显示设备、人体姿态检测方法及应用，所述显示设备包括：摄像头，用于采集图像；显示器，用于显示用户界面；外接输入设备，用于输入在所述用户界面播放的当前视频；控制器被配置为：基于所述摄像头获取的采集图像判定用户在换台时，获取所述用户界面的截图；根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数。

Description

显示设备、人体姿态检测方法及应用
本申请要求于2020年11月12日提交的、申请号为202011262476.4；于2020年11月13日提交的、申请号为202011267208.1；于2021年8月20日提交的、申请号为202110960378.6的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及显示设备技术，具体而言，涉及一种显示设备、人体姿态检测方法及应用。
背景技术
显示设备的功能越来越丰富，更多的显示设备配置有摄像头等图像采集设备，通过摄像头获取用户图像，并配合“肢体运动”应用程序，使显示设备能够实时显示用户身体图像。当用户肢体动作发生变化时，应用程序会也会显示变化后的图像，并通过检测程序对肢体动作的形态进行检测、矫正，达到运动指导的效果。
发明内容
本申请一些实施例提供一种显示设备，包括：摄像头，用于在检测区域内采集躲避球游戏中的目标用户、背景图像；显示器，用于显示包含所述目标用户、背景图像的躲避球游戏用户界面；控制器，被配置为：根据所述背景图像中目标用户的第一尺寸、及所述目标用户在所述用户界面中的第一位置，控制所述用户界面在所述第一位置显示第一标识，其中，所述第一标识用于触发躲避球发射，所述第一标识的尺寸与所述第一尺寸相应；当所述目标用户的上身躯干不完全位于所述采集区域、且所述目标用户正在离开所述采集区域时，控制器控制所述用户界面不再显示所述第一标识。
本申请一些实施例提供一种人体姿态检测的应用，包括：根据躲避球游戏用户界面背景图像中目标用户的第一尺寸、及所述目标用户在所述用户界面中的第一位置，控制所述用户界面在所述第一位置显示第一标识，其中，所述第一标识用于触发躲避球发射，所述第一标识的尺寸与所述第一尺寸相应；当所述目标用户的上身躯干不完全位于摄像头的采集区域、且所述目标用户正在离开所述采集区域时，控制所述用户界面不再显示所述第一标识。
本申请一些实施例提供一种显示设备，包括：摄像头，用于采集图像；显示器，用于显示用户界面；外接输入设备，用于输入在所述用户界面播放的当前视频；控制器，被配置为：基于所述摄像头获取的采集图像判定用户在换台时，获取所述用户界面的截图；根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数。
本申请一些实施例提供一种人体姿态检测的应用，包括：基于获取的采集图像判定用户在换台时，获取用户界面的截图；根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数。
本申请一些实施例提供一种显示设备，包括：显示器，被配置为显示图像画面；与所述显示器连接的控制器，所述控制器被配置为：对待检测视频中的当前帧图像进 行人体姿态检测，确定所述当前帧图像中的人体检测框；确定所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述当前帧图像的前一帧图像中的目标人体检测框是否存在变化；若所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则基于所述前一帧图像中的目标人体检测框，调整所述当前帧图像中的目标人体检测框的大小和/或位置，并将调整后的所述当前帧图像发送至所述显示器进行显示。
本申请一些实施例提供一种人体姿态检测方法，包括：对待检测视频中的当前帧图像进行人体姿态检测，确定所述当前帧图像中的人体检测框；确定所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述当前帧图像的前一帧图像中的目标人体检测框是否存在变化；若所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则基于所述前一帧图像中的目标人体检测框，调整所述当前帧图像中的目标人体检测框的大小和/或位置，并对调整后的所述当前帧图像进行显示。
附图说明
图1为根据本申请一个或多个实施例的显示设备与控制装置之间操作场景的示意图；
图2为根据本申请一个或多个实施例的显示设备200的硬件配置框图；
图3为根据本申请一个或多个实施例的控制设备100的硬件配置框图；
图4为根据本申请一个或多个实施例的显示设备200中软件配置示意图；
图5为根据本申请一个或多个实施例的显示设备200中应用程序的图标控件界面显示示意图；
图6为根据本申请一个或多个实施例的人像追踪示意图；
图7-图8为根据本申请一个或多个实施例的AI健身场景示意图；
图9为根据本申请一个或多个实施例的应用程序UI界面；
图10为根据本申请一个或多个实施例的显示设备应用程序界面的示意图；
图11为根据本申请一个或多个实施例的显示设备躲避球游戏用户界面的示意图；
图12为根据本申请一个或多个实施例的显示设备躲避球游戏用户界面的示意图；
图13-14为根据本申请一个或多个实施例的显示设备识别用户肢体的示意图；
图15A-15B为根据本申请一个或多个实施例的显示设备识别用户换台动作的示意图；
图16-图18为根据本申请一个或多个实施例的显示设备的显示界面示意图。
具体实施方式
为使本申请的目的、实施方式和优点更加清楚，下面将结合本申请示例性实施例中的附图，对本申请示例性实施方式进行清楚、完整地描述，显然，所描述的示例性实施例仅是本申请一部分实施例，而不是全部的实施例。
基于本申请描述的示例性实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请所附权利要求保护的范围。此外，虽然本申请中公开内容按照示范性一个或几个实例来介绍，但应理解，可以就这些公开内容的各个方面也可以单独构成一个完整实施方式。需要说明的是，本申请中对于术语的简要说明，仅是为了方便理解接下来描述的实施方式，而不是意图限定本申请的实施方式。除非另有说明， 这些术语应当按照其普通和通常的含义理解。
图1为根据本申请一个或多个实施例的显示设备与控制装置之间操作场景的示意图，如图1所示，用户可通过移动终端300和控制装置100操作显示设备200。控制装置100可以是遥控器，遥控器和显示设备的通信包括红外协议通信、蓝牙协议通信，无线或其他有线方式来控制显示设备200。用户可以通过遥控器上按键，语音输入、控制面板输入等输入用户指令，来控制显示设备200。在一些实施例中，也可以使用移动终端、平板电脑、计算机、笔记本电脑、和其他智能设备以控制显示设备200。
在一些实施例中，移动终端300可与显示设备200安装软件应用，通过网络通信协议实现连接通信，实现一对一控制操作的和数据通信的目的。也可以将移动终端300上显示音视频内容传输到显示设备200上，实现同步显示功能显示设备200还与服务器400通过多种通信方式进行数据通信。可允许显示设备200通过局域网(LAN)、无线局域网(WLAN)和其他网络进行通信连接。服务器400可以向显示设备200提供各种内容和互动。显示设备200，可以液晶显示器、OLED显示器、投影显示设备。显示设备200除了提供广播接收电视功能之外，还可以附加提供计算机支持功能的智能网络电视功能。
图2示例性示出了根据示例性实施例中控制装置100的配置框图。如图2所示，控制装置100包括控制器110、通信接口130、用户输入/输出接口140、存储器、供电电源。控制装置100可接收用户的输入操作指令，且将操作指令转换为显示设备200可识别和响应的指令，起用用户与显示设备200之间交互中介作用。通信接口130用于和外部通信，包含WIFI芯片，蓝牙模块，NFC或可替代模块中的至少一种。用户输入/输出接口140包含麦克风，触摸板，传感器，按键或可替代模块中的至少一种。
图3示出了根据示例性实施例中显示设备200的硬件配置框图。如图3所示显示设备200包括调谐解调器210、通信器220、检测器230、外部装置接口240、控制器250、显示器260、音频输出接口270、存储器、供电电源、用户接口280中的至少一种。控制器包括中央处理器，视频处理器，音频处理器，图形处理器，RAM，ROM，用于输入/输出的第一接口至第n接口。显示器260可为液晶显示器、OLED显示器、触控显示器以及投影显示器中的至少一种，还可以为一种投影装置和投影屏幕。调谐解调器210通过有线或无线接收方式接收广播电视信号，以及从多个无线或有线广播电视信号中解调出音视频信号，如以及EPG数据信号。检测器230用于采集外部环境或与外部交互的信号。控制器250和调谐解调器210可以位于不同的分体设备中，即调谐解调器210也可在控制器250所在的主体设备的外置设备中，如外置机顶盒等。
在一些实施例中，控制器250，通过存储在存储器上中各种软件控制程序，来控制显示设备的工作和响应用户的操作。控制器250控制显示设备200的整体操作。用户可在显示器260上显示的图形用户界面(GUI)输入用户命令，则用户输入接口通过图形用户界面(GUI)接收用户输入命令。或者，用户可通过输入特定的声音或手势进行输入用户命令，则用户输入接口通过传感器识别出声音或手势，来接收用户输入命令。
在一些实施例中，“用户界面”，是应用程序或操作系统与用户之间进行交互和信息交换的介质接口，它实现信息的内部形式与用户可以接受形式之间的转换。用户界面常用的表现形式是图形用户界面(Graphic User Interface，GUI)，是指采用图形方式显示的与计算机操作相关的用户界面。它可以是在电子设备的显示屏中显示的一个图标、窗口、控件等界面元素，其中控件可以包括图标、按钮、菜单、选项卡、文本框、对话框、状态栏、导 航栏、Widget等可视的界面元素中的至少一种。
图4为根据本申请一个或多个实施例的显示设备200中软件配置示意图，如图4所示，将系统分为四层，从上至下分别为应用程序(Applications)层(简称“应用层”)，应用程序框架(Application Framework)层(简称“框架层”)，安卓运行时(Android runtime)和系统库层(简称“系统运行库层”)，以及内核层。内核层至少包含以下驱动中的至少一种：音频驱动、显示驱动、蓝牙驱动、摄像头驱动、WIFI驱动、USB驱动、HDMI驱动、传感器驱动(如指纹传感器，温度传感器，压力传感器等)、以及电源驱动等。
图5为根据本申请一个或多个实施例的显示设备200中应用程序的图标控件界面显示示意图，如图5中所示，应用程序层包含至少一个应用程序可以在显示器中显示对应的图标控件，如：直播电视应用程序图标控件、视频点播应用程序图标控件、媒体中心应用程序图标控件、应用程序中心图标控件、游戏应用图标控件等。直播电视应用程序，可以通过不同的信号源提供直播电视。视频点播应用程序，可以提供来自不同存储源的视频。不同于直播电视应用程序，视频点播提供来自某些存储源的视频显示。媒体中心应用程序，可以提供各种多媒体内容播放的应用程序。应用程序中心，可以提供储存各种应用程序。
<肢体检测技术应用到人像追踪>
下文将本申请发明人于2020年8月21日，申请号：202010849806.3，名称“一种显示设备人像定位方法”；及本申请发明人于2020年12月31日，申请号：202011620179.2，名称“一种显示设备及人像定位方法”的全部内容加入本申请中。
本申请实施例中，图6为根据本申请一个或多个实施例的人像追踪示意图，如图6所示，摄像头231作为一种检测器230可以内置或外接显示设备200上，在启动运行后，摄像头231可以检测图像数据。摄像头231可以通过接口部件与控制器250连接，从而将检测的图像数据发送给控制器250进行处理。为了检测图像，摄像头231可以包括镜头组件和云台组件，镜头组件设置在云台组件上，云台组件可以带动镜头组件进行转动，以便更改镜头组件的朝向。随着镜头组件的不同朝向，镜头组件可以对位于不同位置上的用户进行视频拍摄，从而获取用户图像数据。显然，不同的朝向对应于不同区域的图像采集，当用户在相对于显示器275正前方位置偏左时，可以通过云台组件上的第一转轴带动固定件以及镜头组件向左转动，以使拍摄的图像中，用户人像位置位于画面的中心区域；而当用户躯体成像位置偏下时，可以通过云台组件中的第二转轴带动镜头组件向上转动，以抬高拍摄角度，使用户人像位置位于画面的中心区域。
基于上述摄像头231，可以在显示设备200中设置自动控制程序，从而通过检测用户的位置调整摄像头231中镜头组件的朝向，并且按照一定频率重复对人像位置的检测过程，从而实现对人像位置的追踪，具体包括以下步骤：
检测人像位置。在摄像头231启动运行后，实时拍摄多帧图像，并将拍摄的图像发送给显示设备200的控制器250。启动摄像头231后，控制器250一方面可以根据所启动的应用程序进行图像处理，例如控制显示器275显示该图像；另一方面可以通过调用检测程序对校对图像进行分析，从而确定用户所在的位置。其中，人像位置的检测可以通过图像处理程序完成。即通过实时抓取摄像头231拍摄的图像，检测肢体信息。肢体信息可以包含关键点和包裹肢体的外框，通过检测的关键点和肢体框位置在图像中位置信息，确定人像位置。其中，关键点可以是指人体图像中能够代表人体 特征的一系列点。例如，眼睛、耳朵、鼻子、脖子、肩部、手肘、手腕、腰部、膝关节以及踝关节等。
关键点的确定可以通过图像识别获得，即可以通过分析画面中特征形状，并与预设的模板进行匹配从而确定关键点对应的图像，并获取图像对应的位置，从而获取各关键点对应的位置。其中，位置可以通过图像中距离边界的像素点数量进行表示。可以根据摄像头231的分辨率和可视角度，以图像的左上角为原点，以向右和向下为正方向构建平面直角坐标系，则图像中的各个像素点均能够通过这一直角坐标系进行表示。
当用户所处位置发生改变或者姿态发生变化时，部分关键点的位置将发生变化。随着这种变化的出现，摄像头231采集的图像中人体相对位置也将发生变化。例如，当人体向左移动位置时，将使摄像头231采集的图像中人体位置偏左，不便于进行图像分析处理和实时显示。
因此，在检测人像位置后，还需要对比人像位置与校对图像中的预设区域，从而确定当前人像位置是否在预设区域中。在一些实施例中，人像位置可以通过肢体框中心位置进行表示，而肢体框中心位置可以通过检测的各关键点位置坐标计算获得。例如，通过获取肢体框水平左右两侧的关键点x轴位置坐标，计算肢体框中心位置，即中心位置x轴坐标x 0＝(x 1+x 2)/2。
图7-图8为根据本申请一个或多个实施例的AI健身场景示意图，但在实际应用中，由于受到用户姿态的影响，以及不同应用程序中的需求不同，在部分应用场景下使用中心位置作为人像位置判断的方式并不能获得较好的显示、检测、跟踪效果。为了获得更加准确的人像位置判断，在一些实施例中，AI健身场景为例，如图7、图8所示，识别出多个关键点以后，还可以根据识别出的关键点建立骨骼线示意图形，从而根据骨骼线图形进一步确定人像所在位置。其中，骨骼线可以通过连接多个关键点进行确定。在用户不同的姿态下，骨骼线所呈现的形状也不同。
需要说明的是，通过绘制的骨骼线还可以对根据骨骼线的运动变化规律来动态调整摄像头的拍摄位置。例如，在判断骨骼线运动状态变化过程为从蹲姿状态变化到站立状态，则可以抬高摄像头的视角，以使处于站姿状态的人像也能够处于图像中的合适区域内，即从图7过渡到图8所示的效果。在判断骨骼线运动状态变化过程为从站立状态变化到蹲姿状态，这可以降低摄像头的视角，以使处于蹲姿状态的人像也能够处于图像中的合适区域内，即从图8过渡到图7所示的效果。
< 肢体检测技术应用到多人追踪>
下文将本申请发明人于2020年8月21日，申请号：202010847711.8，名称“一种人脸特征值创建方法、人物锁定追踪方法及显示设备”；及本申请发明人于2021年2月4日，申请号：202110155019.3，名称“一种人脸特征值创建方法、人物锁定追踪方法及显示设备”的全部内容加入本申请中。
在一些实施例中，继续以AI健身场景为例，在健身跟练模式下，需要对锁定的某一人进行跟踪，并对动作进行打点，生成跟练数据，统计跟练结果。如果在手机场景下，人脸或者肢体离屏幕较近，人脸或肢体在屏幕中占比较大，每帧图像检测到的人物图像相对移动距离较小，一般不会产生丢失(出画)现象。而显示设备不同于手机场景，在人与显示设备交互时，人与显示设备的距离一般较远，人脸或肢体的屏占 比较小，前后帧图像内移动距离会较大，比如人在屏幕前快速走动，极易会导致人物的识别丢失，即产生出画现象。由于当前健身功能多基于肢体信息进行肢体跟随，摄像头实现人像跟随功能通常通过识别人脸或肢体等作为识别依据。不管是肢体或者人脸信息，每次识别到均会生成一个ID作为识别的肢体或人脸的标识。但是，当识别丢失后重新识别，即人物出画又重新入画，会重新生成一个新ID信息，造成同一个人的前后ID不一致，这样对于健身或者摄像头追踪特定人物时，产生丢失会造成不可逆，也就无法对同一个人做到追踪的有效性。
本发明实施例提供的一种显示设备，其控制器被配置为下述步骤：
S11、获取摄像头采集的人物图像信息。
由于可同时与显示设备进行交互的人物可为一个或多个，因此，在至少一个人物与显示设备进行交互时，例如，至少一个人物在利用显示设备进行视频通话、AI健身或摄像头人像追踪等时，摄像头实时进行图像采集。摄像头采集到的人物图像信息包括至少一个人物的图像信息，从图像信息中可读取到该人物的肢体动作和面部特征信息等。
S12、识别人物图像信息，确定锁定追踪人物，以及，创建锁定追踪人物的面部特征信息和特定肢体ID信息。
在存在多个人物与显示设备进行交互时，或者，在初始时刻只有一个人物与显示设备进行交互，但在交互过程中又存在其他人物出现在摄像头的拍摄区域范围内时，这种多人交互场景下导致显示设备无法确定以哪一个人物的指令作为控制指令进行响应，即无法确定以哪一个人物作为特定人物进行追踪，因此，需要在进行交互时确定锁定追踪人物。锁定追踪人物是与显示设备进行交互的其中一个人物，后续交互时仅响应该锁定追踪人物产生的指令。为实现对同一个人物进行锁定追踪，需在多个人物与显示设备交互时，选定其中一个人物作为锁定追踪人物。若仅有一个人物与显示设备进行交互，则锁定追踪人物即为该人物。在确定锁定追踪人物时，可根据每个人物是否做出特定动作来进行判断，而人物的动作识别可根据该人物的肢体关键点信息来确定。
S13、将锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，得到锁定追踪人物的人脸特征值。
由于每个人物具有各自的面部特征信息，且不同人物的面部特征信息不同。因此，可由面部特征信息对每个人物进行识别，若识别到相同或相似的面部特征信息，即可将其识别为同一个人物。
由于在通常情况下，在锁定追踪人物走出画面时，其对应的特定肢体ID信息即丢失，即人物出画后删除对应的特定肢体ID信息。若该人物又重新走进画面，则重新生成对应的肢体ID信息，易出现将同一个人识别为两个人的情况。
因此，在一些实施例中，将锁定追踪人物的特定肢体ID信息和面部特征信息建立关联关系，由相互关联的特定肢体ID信息和面部特征信息作为锁定追踪人物的人脸特征值，以对锁定追踪人物进行标识，面部特征信息不会因人物出画而被删除，会一直保存在控制器中。
在一些实施例中，在AI健身场景下，控制器在执行基于人脸特征值对锁定追踪人物进行锁定追踪，被进一步配置为执行下述步骤：
步骤1311、在摄像头应用为AI健身应用时，确定锁定追踪人物为健身人员。
步骤1312、基于健身人员的人脸特征值，持续采集健身人员基于用户界面中呈现的示范视频进行跟练的跟练动作信息。
步骤1313、基于跟练动作信息生成跟练画面，显示在用户界面中，跟练画面显示在示范视频所在画面的一侧。
上述实施例在实现AI健身场景时，同时利用显示设备配置的AI健身功能的人物可为一个或多个，若人物为一个时，该健身的人物即为锁定追踪人物。显示器的用户界面中呈现示范视频，便于健身人员进行跟练。此时，摄像头应用为AI健身应用，AI健身应用调用摄像头始终采集锁定追踪人物的跟练动作，并显示在显示器中的用户界面中。
<肢体检测技术应用到躲避球游戏>
显示设备还能够将人体姿态检测应用到游戏应用中，下文将以显示设备基于肢体识别预测运动趋势的躲避球显示技术方案、及用户界面为例，对显示设备、基于肢体识别的躲避球显示方法进行阐述。
在一些实施例中，图9为根据本申请一个或多个实施例的应用程序UI界面，例如，所述应用程序UI界面包括4个已安装于电视的应用程序，分别是新闻头条、影院点播、AR躲避球、K歌等。通过使用遥控器、语音等方式实现在显示屏上移动焦点，可以选择不同的应用程序、或其他功能按钮。
在一些实施例中，电视显示屏在展示应用程序UI界面的同时，还被配置为可以展示其他交互元素，所述交互元素可包括例如电视主页控件、搜索控件、消息按钮控件、邮箱控件、浏览器控件、收藏夹控件、信号栏控件等。
为提高电视UI的便利性、形象性，在一些实施例中，本申请实施例中显示设备的控制器响应于对所述交互元素的操作而对电视的UI进行控制。例如，用户通过遥控器等控制器对搜索控件进行点击，可以将搜索UI展示于其它UI的顶部，即控制交互元素映射的应用组件的UI能够变大、或全屏运行和显示。
在一些实施例中，所述交互元素还可通过传感器进行操作，所述传感器可以是但不限于是声学输入传感器，例如麦克风，其可检测包括所需交互元素指示的语音命令。例如，用户可使用“躲避球”或任意其它合适的标识识别所需交互元素，例如搜索控件，并且还可描述将执行与所需交互元素相关的所需动作。控制器可识别语音命令并向UI或其处理部件或引擎提交表征交互的数据。
在一些实施例中，图10为根据本申请一个或多个实施例的显示设备应用程序界面的示意图。参考图10，用户可通过遥控器控制显示屏的焦点，选择AR躲避球应用程序，以使得其图标在所述显示屏的用户界面中高亮度显示；然后通过点击所述高亮度图标，可实现打开图标映射的应用程序。
在一些实施例中，图11为根据本申请一个或多个实施例的显示设备躲避球游戏用户界面的示意图。参考图11，本申请提供的显示设备包括摄像头、显示器、控制器。其中，摄像头用于在检测区域内采集躲避球游戏中的目标用户、背景图像；显示器用于显示包含所述目标用户、背景图像的躲避球游戏用户界面；控制器被配置为：根据所述背景图像中目标用户的第一尺寸、及所述目标用户在所述用户界面中的第一位置，控制所述用户界面在所述第一位置显示第一标识，其中，所述第一标识用于触发躲避球发射，所述第一标识的 尺寸与所述第一尺寸相应，如图11所示。
在一些实施例中，当所述目标用户的上身躯干不完全位于所述采集区域、且所述目标用户正在离开所述采集区域时，控制器控制所述用户界面不再显示所述第一标识。
在一些实施例中，摄像头再起监测区域内获取目标用户图像、以及所述目标用户的背景图像，其中，所述目标用户及背景图像都将显示于躲避球游戏的用户界面。控制器根据目标用户在背景图像中的第一尺寸及所述目标用户在游戏用户界面中的第一位置，控制用户界面在所述第一位置显示第一标识。
控制器根据背景图像中目标用户，即正在进行躲避球游戏用户的尺寸确定第一标识的尺寸，即用户在背景图像中的尺寸越大，所述第一标识也越大；用户在背景图像中的尺寸越小，所述第一标识也越小。
当用户由用户界面中显示的第一位置移动至第二位置时，控制器将根据用户在第二位置时的第二尺寸重新确定第二标识的尺寸，并将第二标识显示于游戏用户界面。
在一些实施例中，控制器控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括：当所述目标用户相对于所述摄像头由远到近运动时，所述第一尺寸逐渐变大，所述第一标识的尺寸相应于所述第一标识变大；当所述目标用户相对于所述摄像头由近到远移动时，所述第一尺寸逐渐变小，所述第一标识的尺寸相应于所述第一标识变小。
例如，当用户由第一位置移动至第二位置时，控制器根据目标用户在背景图像中第一尺寸和第二尺寸的变化，可以相应的控制第二标识相较于第一标识放大、或缩小。
在一些实施例中，控制器控制所述第一标识覆盖显示于所述第一位置的目标用户。例如，所述第一标识可以显示为近似矩形框，当控制器定位目标用户在游戏用户界面中的第一位置后，将符合目标用户当时第一尺寸的近似矩形框，即第一标识覆盖显示于目标用户。
在一些实施例中，图12为根据本申请一个或多个实施例的显示设备躲避球游戏用户界面的示意图。参考图12，当目标用户的上身躯干不完全位于采集区域、且所述目标用户正在离开所述采集区域时，控制器控制用户界面不再显示第一标识。
可以理解，目标用户离开游戏时包括：目标用户走向摄像头采集区域但用户上身躯干任处于所述采集区域内、目标用户正在穿过摄像头采集区域边缘且用户部分上身躯干任处于所述采集区域内、目标用户已穿过摄像头采集区域边缘且用户上身躯干完全不处于所述采集区域内。本申请提供的显示设备可识别目标用户正在穿过摄像头采集区域边缘且用户部分上身躯干任处于所述采集区域内，并控制所述用户界面不再第一标识，也不会触发游戏继续发射躲避球，如图中所示的虚线近似矩形框，所述虚线近似矩形框用于方便理解，其并不显示于显示设备的游戏用户界面，所述虚线近似矩形框表示控制器所识别的部分目标用户上身躯干。
在一些实施例中，图13-14为根据本申请一个或多个实施例的显示设备识别用户肢体的示意图。参考图13，控制器控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括所述控制器：将摄像头采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点；根据所述第一定位点、第二定位点之间的第一间隔距离，确定在所述第一位置处显示第一标识的尺寸。
控制器根据图像识别模型，将摄像头采集的目标用户的上身躯干分解成各个点，根据上肢各个点的位置变化判断用户运动趋势，由部分肢体推测整体位置，从而改变第一标识 的位置和大小，其位置即本申请提供的第一位置，其大小即本申请提供的第一标识的尺寸。
例如，控制器将目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点，然后根据所述第一定位点与第二定位点之间的第一间隔距离，例如所述第一间隔距离为10cm，则控制器控制所述用户界面在第一位置处显示宽度为10cm的第一标识，所述第一标识可实施为近似矩形框。
在一些实施例中，控制器通过下面步骤识别摄像头采集图像中的目标用户。首先，每隔预设帧数，从所述采集视频中抽取一单帧图像；对所述单帧图像进行人体检测，确定所述单帧图像中是否包含人体；若所述单帧图像中包含人体，在所述人体的人体框范围内对所述单帧图像进行人脸检测，确定所述人体框范围内是否包含人脸；若所述人体框范围内包含人脸，对所述人脸进行特征提取，得到所述采集视频中的用户的人脸特征；将所述采集视频中的用户的人脸特征与预设的家庭成员人脸特征库进行比对，确定所述采集视频中的用户是否为家庭成员及游戏用户。
在一些实施例中，控制器对摄像头采集图像中的人体，应用人脸检测算法(例如：可变形的组件模型Deformable Part Model)，在检测出的人体框范围内对该单帧图像进行人脸检测，确定人体框范围内是否包含人脸。若人体框范围内包含人脸，对该人脸进行特征提取，得到该用户的人脸特征。将该用户的人脸特征与家庭成员人脸特征库进行比对，确定该用户是否为家庭成员及游戏用户。
在一些实施例中，参考图14，控制器还被配置为：将摄像头采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点，其他关节标识为第三定位点，所述第三定位点位于左手、和/或左肩、和/或脖子、和/或右肩、和/或右手、和/或左腰、和/或右腰；根据追踪所述第一定位点、第二定位点、第三定位点的位置变化，确定目标用户的上身躯干是否处完全于采集区域、且所述目标用户是否正在离开所述采集区域。
例如，将用户的左手标识为点1、左臂肘部标识为点2、左肩标识为点3、脖子标识为点4、右肩标识为点5、右臂肘部标识为点6、右手标识为点7、左腰标识为点8、右腰标识为点9。
其中，第一定位点实施为点2、第二定位点实施为点6，第三定位点可实施为剩余点中的一个、或多个的组合。控制器在采集图像中识别上述多个点、跟踪所述多个点、根据所述多个点，例如根据9个点计算用户的整体位置以得到第一位置；其中，点2和点6之间的间隔距离决定了第一标识显示框的大小，通过所述多个点的位置变化确定目标用户的运动方向，确定目标用户的上身躯干是否处完全于采集区域、且所述目标用户是否正在离开所述采集区域。
在一些实施例中，控制器对从摄像头接收到的采集图像，每隔预设帧数(例如：每隔90帧)从采集视频中抽取一单帧图像。应用图像识别技术中的人体检测算法(例如：卷积姿态机器Convolutional Pose Machine，检测出人体的各个关节点，再将这些关节点组成的范围确定为该人体的人体框)，对该单帧图像进行人体检测，通过多个点的位置变化确定目标用户的运动方向，确定目标用户的上身躯干是否处完全于采集区域、且所述目标用户是否正在离开所述采集区域。
在一些实施例中，家庭成员特征库中存储着各家庭成员的身体特征以及对应的家庭成员标识。其中，家庭成员的身体特征存储着该家庭成员的各个身体特征点的坐标。例如：家庭成员03的右手的坐标为(10，0)、左手的坐标为(-10，0)、右肩的坐标为(5，-10)、左 肩坐标为(-5，-10)等。
在一些实施例中，按照预定连线规则(例如：将左肩与右肩连线，再将右肩与右臂肘部连线，再将右臂肘部与右手连线)，将采集视频中的用户的各个上身躯干特征点顺序连接起来，从而得到该用户的上身边缘图。同理，得到各家庭成员的上身边缘图。
对一家庭成员，通过连线的角度，来确定该用户的上身边缘图各连线与该家庭成员的上身边缘图对应连线的相似度。例如：对家庭成员A，用户的上身边缘图中由左手、左壁肘部、左肩连成的线的角度为27度。目标用户的上身边缘图中由左手、左壁肘部、左肩连成的线的角度为30度。则目标用户的上身边缘图的该连线、与家庭成员A的上身边缘图的该连线的相似度为1-(|27-30|/30)＝0.9。
确定了目标用户的上身边缘图各连线与该家庭成员的上身边缘图对应连线的相似度之后，将该用户的上身边缘图各连线与该家庭成员的上身边缘图对应连线的相似度的均值，确定为该用户的上身特征与该家庭成员的上身特征的相似度。
基于上文中对显示设备基于肢体识别预测运动方向的躲避球显示方案的阐述，本申请还提供了一种基于肢体识别的躲避球显示方法，所述方法包括：根据躲避球游戏用户界面背景图像中目标用户的第一尺寸、及所述目标用户在所述用户界面中的第一位置，控制所述用户界面在所述第一位置显示第一标识，其中，所述第一标识用于触发躲避球发射，所述第一标识的尺寸与所述第一尺寸相应；当所述目标用户的上身躯干不完全位于摄像头的采集区域、且所述目标用户正在离开所述采集区域时，控制所述用户界面不再显示所述第一标识。所述基于肢体识别的躲避球显示方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括：当目标用户相对于摄像头由远到近运动时，所述第一尺寸逐渐变大，所述第一标识的尺寸相应于所述第一标识变大；当目标用户相对于摄像头由近到远运动时，所述第一尺寸逐渐变小，所述第一标识的尺寸相应于所述第一标识变小。所述基于肢体识别的躲避球显示方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括：将采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点；根据所述第一定位点、第二定位点之间的第一间隔距离，确定在所述第一位置处显示第一标识的尺寸。所述基于肢体识别的躲避球显示方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，所述方法还包括：将采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点，其他关节标识为第三定位点，所述第三定位点位于左手、和/或左肩、和/或脖子、和/或右肩、和/或右手、和/或左腰、和/或右腰；根据追踪所述第一定位点、第二定位点、第三定位点的位置变化，确定目标用户的上身躯干是否处完全于采集区域、且所述目标用户是否正在离开所述采集区域。所述基于肢体识别的躲避球显示方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，控制所述用户界面在所述第一位置显示第一标识，具体包括所述控制器：将所述第一标识覆盖显示于所述第一位置的目标用户。所述基于肢体识别的躲避球显示方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
本申请实施例的有益效果在于，通过构建第一尺寸，可以控制第一标识的尺寸；进一步通过构建第一位置，可以实现第一标识的位置获取；进一步通过控制第一标识不显示，可以实现用户离开游戏时不发射躲避球；通过构建第一定位点、第二定位点、第三定位点，可以实现识别游戏用户肢体、预测游戏用户运动趋势、根据用户在屏幕中显示的大小调整第一标识的尺寸、实现用户不在屏幕显示范围内时不进行游戏。
< 肢体检测技术应用到换台>
在一些实施例中，控制器基于摄像头获取的采集图像判定用户在换台，具体包括：
控制器以第一频率分析摄像头获取的采集图像以识别用户存在。具体的，显示设备开机后，控制器控制摄像头采集图像并获取其图像预览，通过对预览图像使用图像识别算法以检测是否有用户在观看显示设备。在控制器检测到用户之前，控制器以第一频率分析采集图像，例如控制器对摄像头采集的图像以每10帧的间隔进行图像识别分析，以检测是否有用户存在于摄像头的检测范围，通过第一频率的设置，可以有效的节约控制器的计算资源。
识别到用户后，控制器以第二频率分析所述采集图像以识别用户的上肢运动，当识别用户肘部、手部位置发生变化、且手部位置与肘部位置高度差小于高度阈值时，识别用户在进行换台动作，所述第二频率大于第一频率。图15A-15B为根据本申请一个或多个实施例的显示设备识别用户换台动作的示意图，在一些实施例中，摄像头获取关于用户的采集图像后，控制器将抓取的数据帧发送至图像识别模型进行分析以判定用户是否进行了换台动作。图像识别模型将用户的上肢，即胳膊、躯体进行线条、节点标识，例如将肘部标识为点1、手部标识为点2，在用户换台动作之前，其相对位置如图15A所示。在一些实施例中，用户在使用遥控器换台，即切换频道、或切换第一片源为第二片源时，通常会有一个抬起胳膊、指向显示设备的动作发生；控制器通过分析摄像头采集图像中数据帧，使用图像识别模型对其进行识别，当表示肘部的点1和手部的点2位置都发生变化、手部点2位置升高、且点1和点2位置接近平行时，即手部位置点2与肘部位置点1的高度差小于高度阈值时，例如所述高度阈值实施为5cm，控制器判定、识别用户此动作有很大的可能为用户拿起遥控器切换第一片源为第二片源，即进行了换台动作；
此时，将触发控制器发送包含所述第二片源截图的第一指令至服务器，所述第一指令用于使所述服务器根据所述截图识别所述第二片源以确定是否可以提供优化图像参数、及优化声音参数；在服务器可提供所述优化图像参数、及优化声音参数时，控制器控制所述用户界面播放所述优化图像参数、及优化声音参数配置的第二片源，如图15B所示。
在一些实施例中，显示设备识别用户肢体进行换台操作步骤如下：
在步骤901中，显示设备检测到用户时加快预览分析频率。显示设备控制器基于摄像头获取的采集图像判定用户在换台，具体包括所述控制器：基于所述采集图像识别用户的换台动作后，判定第一时间长度内所述显示设备音频输出是否有瞬间静音；如果是，判定用户换台；否则，判定用户没有换台。例如，显示设备开机后控制器检测到用户之前，控制器以第一频率分析采集图像以识别是否有用户在观看显示设备。例如，控制器对摄像头采集的图像以每10帧的间隔进行图像识别分析，以检测是否有用户存在于摄像头的检测范围，通过第一频率的设置，可以有效的节约控制器的计算资源。
在步骤902中，显示设备进行用户行为分析。识别到用户后，控制器以第二频率分析所述采集图像进行用户行为分析，例如可实施为识别用户的上肢运动。
在步骤903中，检测到用户抬手动作时做标记。通过分析采集图像，控制器识别用户肘部、手部位置发生变化时做标记。例如其手部位置与肘部位置高度差小于高度阈值，控制器判定用户进行了疑似换台操作。
在步骤904中，显示设备对收到声音变化且间隔时间小于30秒进行判定。控制器判定自做标记起，即自识别用户发生了疑似换台操作后，控制器判定在第一时间长度内显示设备音频输出是否有瞬间静音，所述瞬间静音通常发生在显示设备换台、或切换片源时，所述第一时间长度例如可实施为30秒。如果控制器判定在第一时间长度，即上述30秒内显示设备的音频输出有瞬间静音发生，则判定用户切换第一片源为第二片源；此时，将触发控制器发送包含所述第二片源截图的第一指令至服务器，所述第一指令用于使所述服务器根据所述截图识别所述第二片源以确定是否可以提供优化图像参数、及优化声音参数；在服务器可提供所述优化图像参数、及优化声音参数时，控制器控制所述用户界面播放所述优化图像参数、及优化声音参数配置的第二片源。需要说明的是，根据换台后的获取的截图识别当前播放视频，也可以在显示设备本地进行实现。
如果控制器判定在第一时间长度，即上述30秒内显示设备的音频输出没有瞬间静音发生，则判定用户没有切换第二片源，丢弃所分析的数据，如步骤904-1所示。
在一些实施例中，控制根据截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数，具体包括所述控制器：向服务器发送包含所述截图的第一指令，所述第一指令用于使所述服务器根据所述截图识别所述当前视频以确定可以提供的优化图像参数、及优化声音参数；接收来自服务器发送的包含优化图像参数、及优化声音参数的第二指令；根据所述第二指令，为所述用户界面播放的当前视频配置所述优化图像参数、及优化声音参数。例如，显示设备的控制器在识别、判定用户拿起遥控器切换用户界面播放的第一片源为第二片源时，将对当前用户界面播放的第二片源进行截图；然后控制器将所述截图通过第一指令发送至服务器。
在一些实施例中，截图为当前视频在所述用户界面角位置处的显示设备台标截图、和/或当前视频在所述用户界面边位置处的文本截图。截图为第二片源在用户界面角位置处的显示设备台标截图、和/或所述第二片源在所述用户界面边位置处的文本截图。例如，当用户在观看直播频道时，通常显示设备台标位于显示设备屏幕的左上角位置，控制器通过对所述显示设备台标截图供服务器识别可以提高图像识别的效率，减小数据发送量；当用户在观看影视综艺节目时，通常其节目名称会位于显示设备屏幕的上边、或下边、或侧边位置，控制器通过对屏幕边缘位置处的文本信息进行截图供服务器识别可以提高图像识别的效率，减小数据发送量，如步骤904-2所示。
基于上文中对显示设备自动配置视频参数方案的阐述，本申请还提供了一种显示设备端自动配置视频参数的方法，所述方法包括：基于获取的采集图像判定用户在换台时，获取用户界面的截图；根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数。所述自动配置视频参数方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，基于获取的采集图像判定用户在换台，具体包括：基于所述采集图像识别用户的换台动作后，判定第一时间长度内音频输出是否有瞬间静音；如果是，判定用户换台；否则，判定用户没有换台。所述自动配置视频参数方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，基于所述采集图像判定用户在换台，具体包括：以第一频率分析所述采集图像以识别用户存在；识别用户后以第二频率分析所述采集图像以识别用户的上肢运动，当识别用户肘部、手部位置发生变化、且手部位置与肘部位置高度差小于高度阈值时，识别用户在进行换台动作，所述第二频率大于第一频率。所述自动配置视频参数方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数，具体包括：向服务器发送包含所述截图的第一指令，所述第一指令用于使所述服务器根据所述截图识别所述当前视频以确定可以提供的优化图像参数、及优化声音参数；接收来自服务器发送的包含优化图像参数、及优化声音参数的第二指令；根据所述第二指令，为所述用户界面播放的当前视频配置所述优化图像参数、及优化声音参数。所述自动配置视频参数方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
在一些实施例中，所述截图为当前视频在所述用户界面角位置处的显示设备台标截图、和/或当前视频在所述用户界面边位置处的文本截图。所述自动配置视频参数方法的具体操作、步骤在上文对显示设备实现方案中已进行详细阐述，在此不再赘述。
基于上文中对服务器实现自动配置视频参数方案的阐述，本申请还提供了一种用于服务器端的自动配置视频参数，所述方法包括：接收显示设备发送的包含第二片源截图的第一指令；基于所述第一指令包含的所述截图识别所述第二片源的名称，以确定是否可以提供所述第二片源对应的优化图像参数、及优化声音参数；可提供所述优化图像参数、及优化声音参数时，发送包含所述优化图像参数、及优化声音参数的第二指令至所述显示设备。所述服务器实现自动配置视频参数的具体操作、步骤在上文对服务器实现方案中已进行详细阐述，在此不再赘述。
本申请实施例的有益效果在于，通过摄像头采集用户图像，可以实现即时检测用户换台的肢体动作；进一步通过获取用户界面截图，可以实现当前播放视频的识别；进一步通过对识别视频配置优化声音图像参数，在显示设备通过HDMI连接机顶盒作为显示器使用时，可以实现提高识别用户换台的准确率、识别节目信息、及时自动配置换台后的播放参数。
<肢体检测精度提升算法>
在一些实施例中，继续以AI健身场景为例进行举例说明，本身一些实施例可以提高人体姿态的检测精度。图16-图18为根据本申请一个或多个实施例的显示设备的显示界面示意图，参照图16，左侧区域为教练动作展示图像，右侧区域为摄像头实时采集的用户体态图像。当用户出现在屏幕中时，首先检测到用户所在区域为实线圈所在区域，之后检测用户的动作，通过比对用户动作与教练动作是否一致，得到用户的准确率并显示在当前显示界面上。
其中，在人体姿态检测中，前后两帧图片中的任何微小差别，如光线、帧率、背景等，都会导致检测结果产生差别，在可视化显示过程中人体检测框会存在抖动，进而影响用户体验。
考虑上述技术问题，本申请实施例中提供一种显示设备，可以防止人体姿态检测结果中的人体检测框，因前后相邻两帧图片中的微小差别而出现抖动，从而提升可视化显示效果。下面采用详细的实施例进行详细说明。
在本申请一些实施例中，显示设备200可以通过摄像头201采集用户的运动视频，并将采集到的运动视频作为待检测视频发送至控制器250进行处理。
其中，控制器250在接收到上述待检测视频之后，利用人体姿态检测网络模型对待检测视频中的每一帧图像进行人体姿态检测，确定每一帧图像中与人物相关的信息，该信息包括人物的位置信息，包括人体检测框的中心点位置(x c,y c)、宽度(w c)、高度(h c)；以及人体姿态信息，包括人体关节点位置信息(x loc,y loc)。
其中，人体检测框的中心点位置(x c,y c)可以用于确定人体检测框的位置，人体检测框的宽度(w c)与高度(h c)可以用于确定人体检测框的大小。
控制器250在得到每一帧图像的人体姿态检测结果后，基于当前帧图像中的目标人体检测框的中心点位置，以及前一帧图像中的目标人体检测框的中心点位置，确定当前帧图像中的目标人体检测框的位置相较于前一帧图像中的目标人体检测框是否存在变化；和/或，基于当前帧图像中的目标人体检测框的宽度与高度，以及前一帧图像中的目标人体检测框的宽度与高度，确定当前帧图像中的目标人体检测框的大小相较于前一帧图像中的目标人体检测框是否存在变化。
若当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，则判断该变化的变化幅度是否小于预设变化阈值。
如果上述变化的变化幅度小于预设变化阈值，则可以认为上述变化并不是由于用户的姿态变化而引起的，而是由前后两帧图片中的某些微小差别导致。此种情况下，可以基于前一帧图像中的目标人体检测框，调整当前帧图像中的目标人体检测框的大小和/或位置，使得当前帧图像中的目标人体检测框的大小和/或位置与前一帧图像中的目标人体检测框相同，之后将调整后的当前帧图像发送至显示器进行显示。由于调整之后的当前帧图像中的目标人体检测框的大小和/或位置与前一帧图像中的目标人体检测框相同，因此，在对当前帧图像进行可视化显示时，可以有效避免人体姿态检测结果中的人体检测框因前后相邻两帧图片中的微小差别而出现抖动，提升可视化显示效果。
如果上述变化的变化幅度大于或等于上述预设变化阈值，则可以认为上述变化是由于用户的姿态变化而引起的，此种情况下，则可以直接将当前帧图像发送至显示器进行显示。此时，用户即可观察到显示屏幕中显示的人体检测框会随着用户姿态变化而变化，保障人体姿态检测结果可视化显示的准确度。
本申请实施例所提供的显示设备，通过比较当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的大小和/或位置，可以确定当前帧图像中的目标人体检测框的大小和/或位置是否存在变化，在当前帧图像中的目标人体检测框的大小和/或位置存在变化，且变化幅度小于预设变化阈值时，则基于前一帧图像中的目标人体检测框调整当前帧图像中的目标人体检测框的大小和/或位置，并将调整后的当前帧图像发送至显示器进行显示，从而可以防止人体姿态检测结果中的人体检测框，因前后相邻两帧图片中的微小差别而出现抖动，提升可视化显示效果。
基于上述实施例中描述发内容，在本申请一些实施例中，当检测场景中同时存在多人，且人数多于检测场景所需人数(例如单人健身时需要且仅需检测一个人)时，目标人体检测框会在多个人之间来回切换，呈现出人体检测框漂移的现象。为了防止目标人体检测框出现漂移的现象，本申请实施例中可以采用人体检测框过剩策略，在人体姿态检测过程中建立多于需求人数的人体检测框。
其中，在当前帧图像中包括两个或两个以上人体检测框时，分别确定当前帧图像中的各个人体检测框与前一帧图像中的目标人体检测框的IOU(Intersection-over-Union，交并比)；将当前帧图像中与前一帧图像中的目标人体检测框的IOU最大的人体检测框，确定为当前帧图像中的目标人体检测框。
在一种可行的实施方式中，可以利用当前帧图像中的各个人体检测框的中心点位置(x c,y c)、宽高信息(w c,h c)，以及前一帧图像中的目标人体检测框的中心点位置(x c,y c)、宽高信息(w c,h c)，来分别计算当前帧图像中的各个人体检测框与前一帧图像中的目标人体检测框的IOU。
可以理解的是，在可视化显示中，帧率因素主要影响目标人体检测框显示的准确度，因此，本申请实施例中可以设置一个与帧率有关的IOU阈值，来降低帧率对目标人体检测框的影响，从而可以适应不同的图像捕捉设备和显示设备。
在一种可行的实施方式中，该IOU阈值为可以通过公式1计算，
Figure PCTCN2021117797-appb-000001
其中，k为可调节常数，θ为待检测视频的帧率。
在一些实施例中，
Figure PCTCN2021117797-appb-000002
在一些实施例中，在确定出当前帧图像中的目标人体检测框之后，确定当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的交并比IOU是否大于预设IOU阈值；如果当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的IOU大于预设IOU阈值，则判断当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框是否存在变化；如果当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的IOU小于或等于上述预设IOU阈值，则输出提示信息，该提示信息用于提示用户当前帧图像中的人体已移出。
在一些实施例中，可以设置一个变化阈值，来判断目标人体检测框是否存在抖动。
其中，在判断当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框是否存在抖动时，如果当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则确定目标人体检测框存在抖动，此时基于前一帧图像中的目标人体检测框，调整当前帧图像中的目标人体检测框的大小和/或位置，并将调整后的当前帧图像发送至显示器进行显示。如果当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框不存在变化，或者当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，且变化幅度大于或等于上述预设变化阈值，则确定目标人体检测框不存在抖动，此时将当前帧图像发送至显示器进行显示。
其中，当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框的变化幅度δ可以通过公式2确定：
Figure PCTCN2021117797-appb-000003
其中，I t表示前帧图像中的目标人体检测框的大小和/或位置信息，I t-1表示前一帧图像中的目标人体检测框的大小和/或位置信息。
可以理解的是，由于不同摄像采集设备采集图像的帧率不同，不同显示设备支持的帧 率也不同。当帧率较大时，图像显示流畅，上下帧之间间隔时间较短，两帧图像本身差别不大，因此上述变化幅度较小；而当帧率较小时，图像显示不流畅，上下帧之间间隔时间较长，两帧图像本身差别较大，因此上述变化幅度较大。在这两种情况下，为了调整该变化阈值避免误判，可以设置一个与帧率有关的变化阈值，来判断目标人体检测框是否存在抖动。
在一种可行的实施方式中，按照公式3来设置上述预设变化阈值；
Figure PCTCN2021117797-appb-000004
其中，α为可调节常数，θ为待检测视频的帧率。
在一些实施例中，
Figure PCTCN2021117797-appb-000005
在一些实施例中，参考图17，假设图17中的实线框为当前帧图像中的目标人体检测框，虚线框为前一帧图像中的目标人体检测框，则由于当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，因此，当上述变化幅度小于预设变化阈值时，反馈给用户的直观感受就是显示界面中的目标人体检测框存在抖动。
为了避免上述抖动，本申请实施例中，若当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则基于前一帧图像中的目标人体检测框，调整当前帧图像中的目标人体检测框的大小和/或位置，使当前帧图像中的目标人体检测框的大小和/或位置与前一帧图像中的目标人体检测框相同。再参考图18，由于当前帧图像中调整后的目标人体检测框的大小和/或位置与前一帧图像中的目标人体检测框相同，从而反馈给用户的直观感受就是显示界面中的目标人体检测框保持静止。
本申请实施例所提供的显示设备，基于待检测视频的帧率，来设置上述IOU阈值与上述变化阈值，在判断人体检测框是否抖动时，可以有效降低帧率对判断结果的影响，既可以防止人体检测框出现抖动，还能够保障人体检测框的可视化显示准确度。
基于上述实施例中所描述的内容，在一些实施例中，由于人体姿态检测模型的不稳定性以及光照因素，可视化显示过程中还可能会存在关节点抖动的问题。
在一些实施例中，为了防止可视化显示过程中关节点抖动的问题，可以利用浮点化方法，进行人体关节点位置坐标校准，并基于光照参数，来改善人体关节点显示的稳定性。
在一种可行的实施方式中，可以通过以下公式，将当前帧图像中的人体关节点位置(x loc,y loc)转换为浮点型人体关节点位置(x f,y f)，如公式4：
Figure PCTCN2021117797-appb-000006
其中，i＝0、1、…、s-1、s，表示当前帧图像进行人体姿态检测后获得的热图中的尺寸；v i,j表示所述热图中坐标(x i,y j)处的值；γ∈(0,1)，表示光照参数。
其中，不同光照条件下γ的取值不同，当光线充足时，v i,j整体较大，此时应当设置γ较大，而当光照不足或过强时，应设置γ较小，如0.05。
在本实施例中，在确定浮点型人体关节点位置(x f,y f)后，根据浮点型人体关节点位置 (x f,y f)，在当前帧图像中标注待显示人体关节点，并将标注有待显示人体关节点的当前帧图像发送至显示器进行显示。
本申请实施例所提供的显示设备，基于光照参数，对当前帧图像中的人体关节点信息进行浮点化处理，可以有效防止人体姿态检测结构在可视化显示时，人体关节点存在抖动的问题，提升可视化展示效果，
本申请一些实施例中还提供了一种人体姿态检测方法，该方法包括：
S601、对待检测视频中的当前帧图像进行人体姿态检测，确定当前帧图像中的人体检测框。
S602、确定当前帧图像中的目标人体检测框的大小和/或位置相较于当前帧图像的前一帧图像中的目标人体检测框是否存在变化。
S603、若当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则基于前一帧图像中的目标人体检测框，调整当前帧图像中的目标人体检测框的大小和/或位置，并对调整后的当前帧图像进行显示。
本申请实施例所提供的人体姿态检测方法，通过比较当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的大小和/或位置，可以确定当前帧图像中的目标人体检测框的大小和/或位置是否存在变化，在当前帧图像中的目标人体检测框的大小和/或位置存在变化，且变化幅度小于预设变化阈值时，则基于前一帧图像中的目标人体检测框调整当前帧图像中的目标人体检测框的大小和/或位置，并对调整后的当前帧图像进行显示，从而可以防止人体姿态检测结果中的人体检测框，因前后相邻两帧图片中的微小差别而出现抖动，提升可视化显示效果。
基于上述实施例，在本申请一些实施例中，在当前帧图像中包括两个或两个以上人体检测框时，分别确定当前帧图像中的各个人体检测框与前一帧图像中的目标人体检测框的IOU；将当前帧图像中与前一帧图像中的目标人体检测框的IOU最大的人体检测框，确定为当前帧图像中的目标人体检测框。
在本申请实施例中，假设当前帧图像中包括两个人体检测框，分别为人体检测框1与人体检测框2，则先确定人体检测框1与前一帧图像中的目标人体检测框的IOU1，以及人体检测框2与前一帧图像中的目标人体检测框的IOU2，如果IOU1＞IOU2，则将人体检测框1作为当前帧图像中的目标人体检测框；如果IOU2＞IOU1，则将人体检测框2作为当前帧图像中的目标人体检测框。
在一种可行的实施方式中，在确定当前帧图像中的目标人体检测框之后，判断当前帧图像中人体是否已移出。
其中，确定当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的交并比IOU是否大于预设IOU阈值；如果当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的IOU大于预设IOU阈值，则确定当前帧图像中人体未移出；如果当前帧图像中的目标人体检测框与前一帧图像中的目标人体检测框的IOU小于或等于上述预设IOU阈值，则确定当前帧图像中人体已移出。
在一些实施例中，按照公式5来设置上述预设IOU阈值，
Figure PCTCN2021117797-appb-000007
其中，k可调节常数，θ为待检测视频的帧率。
在一些实施例中，如果确定当前帧图像中人体已移出，则输出提示信息，该提示信息用于提示用户当前帧图像中的人体已移出；如果确定当前帧图像中人体未移出，则继续判断当前帧图像中的目标人体检测框是否存在变化。
在一种可行的实施方式中，分别确定当前帧图像中的目标人体检测框的检测框信息，以及前一帧图像中的目标人体检测框的检测框信息，该检测框信息包括人体检测框的中心点坐标信息、宽度信息及长度信息。
确定当前帧图像中的目标人体检测框的检测框信息与前一帧图像中的目标人体检测框的检测框信息之间的差异值；当该差异值为零时，确定当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框不存在变化；当该差异值不为零时，确定当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化。
在一种可行的实施方式中，当确定当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框存在变化时，根据上述差异值确定当前帧图像中的目标人体检测框的大小和/或位置的变化幅度。如果该变化幅度小于预设变化阈值，则基于前一帧图像中的目标人体检测框，调整当前帧图像中的目标人体检测框的大小和/或位置，并将调整后的当前帧图像发送至显示器进行显示；否则，将当前帧图像发送至显示器进行显示。
在一种可行的实施方式中，当确定当前帧图像中的目标人体检测框的大小和/或位置相较于前一帧图像中的目标人体检测框不存在变化，或者当前帧图像中的目标人体检测框的大小和/或位置的变化幅度大于或等于上述预设变化阈值时，将当前帧图像发送至显示器进行显示。
在一些实施例中，按照公式6来设置上述预设变化阈值
Figure PCTCN2021117797-appb-000008
其中，α为可调节常数，θ为待检测视频的帧率。
本申请实施例所提供的人体姿态检测方法，基于待检测视频的帧率，来设置上述IOU阈值与上述变化阈值，从而在判断人体检测框是否抖动时，可以有效降低帧率对判断结果的影响，既可以防止人体检测框出现抖动，还能够保障人体检测框的可视化显示准确度。
基于上述实施例中所描述的内容，在一些实施例中，在显示当前帧图像之前，可以通过以下公式7，将当前帧图像中的人体关节点位置(x loc,y loc)转换为浮点型人体关节点位置(x f,y f)：
Figure PCTCN2021117797-appb-000009
其中，i＝0、1、…、s-1、s，表示当前帧图像进行人体姿态检测后获得的热图中的尺寸；v i,j表示所述热图中坐标(x i,y j)处的值；γ∈(0,1)，表示光照参数。
其中，不同光照条件下γ的取值不同，当光线充足时，v i,j整体较大，此时应当设置γ 较大，而当光照不足或过强时，应设置γ较小。
本申请实施例所提供的人体姿态检测方法，基于光照参数，对当前帧图像中的人体关节点信息进行浮点化处理，可以有效防止人体姿态检测结构在可视化显示时，人体关节点存在抖动的问题，提升可视化展示效果。
为了方便解释，已经结合具体的实施方式进行了上述说明。但是，上述在一些实施例中讨论不是意图穷尽或者将实施方式限定到上述公开的具体形式。根据上述的教导，可以得到多种修改和变形。上述实施方式的选择和描述是为了更好的解释原理以及实际的应用，从而使得本领域技术人员更好的使用实施方式以及适于具体使用考虑的各种不同的变形的实施方式。

Claims (12)

  1. 一种显示设备，包括：
    显示器，用于显示用户界面；
    外接输入设备，用于输入在所述用户界面播放的当前视频；
    控制器，被配置为：
    基于摄像头获取的采集图像判定用户在换台时，获取所述用户界面的截图；
    根据所述截图为用户界面播放的当前视频配置优化图像参数、及优化声音参数。
  2. 如权利要求1所述显示设备，所述控制器被配置为：
    在基于所述摄像头获取的采集图像判定用户在换台时，基于所述采集图像识别用户的换台动作后，判定第一时间长度内所述显示设备音频输出是否有瞬间静音；
    如果是，判定用户换台；否则，判定用户没有换台。
  3. 如权利要求1所述显示设备，所述控制器被配置为：
    当基于所述采集图像判定用户在换台时，以第一频率分析所述采集图像以识别用户存在；
    识别用户后所述控制器以第二频率分析所述采集图像以识别用户的上肢运动，当识别用户肘部、手部位置发生变化、且手部位置与肘部位置高度差小于高度阈值时，识别用户在进行换台动作，所述第二频率大于第一频率。
  4. 如权利要求1所述显示设备，所述控制器被配置为：
    向服务器发送包含所述截图的第一指令，所述第一指令用于使所述服务器根据所述截图识别所述当前视频以确定可以提供的优化图像参数、及优化声音参数；接收来自服务器发送的包含优化图像参数、及优化声音参数的第二指令；
    根据所述第二指令，为所述用户界面播放的当前视频配置所述优化图像参数、及优化声音参数。
  5. 如权利要求1所述显示设备，所述截图为当前视频在所述用户界面角位置处的显示设备台标截图、和/或当前视频在所述用户界面边位置处的文本截图。
  6. 一种人体姿态检测方法，包括：
    对待检测视频中的当前帧图像进行人体姿态检测，确定所述当前帧图像中的人体检测框；
    确定所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述当前帧图像的前一帧图像中的目标人体检测框是否存在变化；
    若所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述前一帧图像中的目标人体检测框存在变化，且变化幅度小于预设变化阈值，则基于所述前一帧图像中的目标人体检测框，调整所述当前帧图像中的目标人体检测框的大小和/或位置，并对调整后的所述当前帧图像进行显示。
  7. 如权利要求6所述的方法，包括：
    在所述当前帧图像中包括两个或两个以上人体检测框时，分别确定所述当前帧图像中的各个人体检测框与所述前一帧图像中的目标人体检测框的交并比IOU；
    将所述当前帧图像中与所述前一帧图像中的目标人体检测框的IOU最大的人体检测框，确定为所述当前帧图像中的目标人体检测框。
  8. 如权利要求7所述的方法，包括：
    确定所述当前帧图像中的目标人体检测框与所述前一帧图像中的目标人体检测框的交并比IOU是否大于预设IOU阈值；
    当所述当前帧图像中的目标人体检测框与所述前一帧图像中的目标人体检测框的IOU大于所述预设IOU阈值时，确定所述当前帧图像中的目标人体检测框的大小和/或位置相较于所述前一帧图像中的目标人体检测框是否存在变化。
  9. 一种人体姿态检测的应用，包括：
    根据躲避球游戏用户界面背景图像中目标用户的第一尺寸、及所述目标用户在所述用户界面中的第一位置，控制所述用户界面在所述第一位置显示第一标识，其中，所述第一标识用于触发躲避球发射，所述第一标识的尺寸与所述第一尺寸相应；
    当所述目标用户的上身躯干不完全位于摄像头的采集区域、且所述目标用户正在离开所述采集区域时，控制所述用户界面不再显示所述第一标识。
  10. 如权利要求9所述应用，控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括：
    当目标用户相对于摄像头由远到近运动时，所述第一尺寸逐渐变大，所述第一标识的尺寸相应于所述第一标识变大；
    当目标用户相对于摄像头由近到远运动时，所述第一尺寸逐渐变小，所述第一标识的尺寸相应于所述第一标识变小。
  11. 如权利要求9所述应用，控制所述用户界面在所述第一位置显示第一标识，所述第一标识的尺寸与所述第一尺寸相应，具体包括：
    将采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点；
    根据所述第一定位点、第二定位点之间的第一间隔距离，确定在所述第一位置处显示第一标识的尺寸。
  12. 如权利要求11所述的应用，包括：
    将采集图像中目标用户的左臂肘部标识为第一定位点、右臂肘部标识为第二定位点，其他关节标识为第三定位点，所述第三定位点位于左手、和/或左肩、和/或脖子、和/或右肩、和/或右手、和/或左腰、和/或右腰；
    根据追踪所述第一定位点、第二定位点、第三定位点的位置变化，确定目标用户的上身躯干是否处完全于采集区域、且所述目标用户是否正在离开所述采集区域。
PCT/CN2021/117797 2020-11-12 2021-09-10 显示设备、人体姿态检测方法及应用 Ceased WO2022100262A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202180057495.XA CN116114250B (zh) 2020-11-12 2021-09-10 显示设备、人体姿态检测方法及应用

Applications Claiming Priority (6)

Application Number Priority Date Filing Date Title
CN202011262476.4A CN112543359B (zh) 2020-11-12 2020-11-12 一种显示设备及自动配置视频参数的方法
CN202011262476.4 2020-11-12
CN202011267208.1 2020-11-13
CN202011267208.1A CN112473121B (zh) 2020-11-13 2020-11-13 一种显示设备及基于肢体识别的躲避球显示方法
CN202110960378.6A CN116246333A (zh) 2021-08-20 2021-08-20 显示设备与人体姿态检测结果的处理方法
CN202110960378.6 2021-08-20

Publications (1)

Publication Number Publication Date
WO2022100262A1 true WO2022100262A1 (zh) 2022-05-19

Family

ID=81600762

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/117797 Ceased WO2022100262A1 (zh) 2020-11-12 2021-09-10 显示设备、人体姿态检测方法及应用

Country Status (2)

Country Link
CN (1) CN116114250B (zh)
WO (1) WO2022100262A1 (zh)

Cited By (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115065844A (zh) * 2022-05-24 2022-09-16 北京跳悦智能科技有限公司 一种主播肢体动作节奏的自适应调整方法
CN115955603A (zh) * 2022-12-06 2023-04-11 广州紫为云科技有限公司 一种基于智慧屏的体感交互的智能摄像头装置和实现方法
CN116129524A (zh) * 2023-01-04 2023-05-16 长沙观谱红外科技有限公司 一种基于红外图像的姿态自动识别系统和方法
CN116489453A (zh) * 2023-03-31 2023-07-25 海信视像科技股份有限公司 显示设备、终端设备及连屏动作评分显示方法
CN116824475A (zh) * 2022-12-09 2023-09-29 慧之安信息技术股份有限公司 一种基于边缘计算的飞机场智能化安全识别方法和系统
CN118097027A (zh) * 2024-04-19 2024-05-28 杭州欣禾圣世科技有限公司 一种基于进化计算的多视频3d人体动作捕捉方法及系统
CN119166089A (zh) * 2024-10-11 2024-12-20 广东魅视科技股份有限公司 一种控制分布式显示设备的方法、装置及可读介质

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101895705A (zh) * 2010-06-25 2010-11-24 中兴通讯股份有限公司 一种获取手机电视频道信息的方法及终端
US20110286631A1 (en) * 2010-05-21 2011-11-24 Qualcomm Incorporated Real time tracking/detection of multiple targets
CN108419098A (zh) * 2018-03-30 2018-08-17 深圳Tcl数字技术有限公司 电视机的控制方法以及电视机
CN110677585A (zh) * 2019-09-30 2020-01-10 Oppo广东移动通信有限公司 目标检测框的输出方法、装置、终端及存储介质
CN111417028A (zh) * 2020-03-13 2020-07-14 腾讯科技（深圳）有限公司 信息处理方法、装置、存储介质及电子设备
CN111754571A (zh) * 2019-03-28 2020-10-09 北京沃东天骏信息技术有限公司 一种姿态识别方法、装置及其存储介质
CN111897430A (zh) * 2020-07-30 2020-11-06 深圳创维－Rgb电子有限公司 应用的控制方法、显示终端及计算机可读存储介质
CN112473121A (zh) * 2020-11-13 2021-03-12 海信视像科技股份有限公司 一种显示设备及基于肢体识别的躲避球显示方法
CN112543359A (zh) * 2020-11-12 2021-03-23 海信视像科技股份有限公司 一种显示设备及自动配置视频参数的方法

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2014525086A (ja) * 2011-07-05 2014-09-25 サウジ アラビアン オイル カンパニー 従業員の健康および生産性の監視および改善のためのフロアマットシステムおよび関連するコンピュータ媒体、ならびにコンピュータにより実行される方法
WO2014119894A1 (en) * 2013-01-29 2014-08-07 Samsung Electronics Co., Ltd. Method of performing function of device and device for performing the method
CN103686450A (zh) * 2013-12-31 2014-03-26 广州华多网络科技有限公司 视频处理方法及系统
CN111866498B (zh) * 2020-07-24 2022-04-12 海信视像科技股份有限公司 一种摄像头异常处理方法及显示设备

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110286631A1 (en) * 2010-05-21 2011-11-24 Qualcomm Incorporated Real time tracking/detection of multiple targets
CN101895705A (zh) * 2010-06-25 2010-11-24 中兴通讯股份有限公司 一种获取手机电视频道信息的方法及终端
CN108419098A (zh) * 2018-03-30 2018-08-17 深圳Tcl数字技术有限公司 电视机的控制方法以及电视机
CN111754571A (zh) * 2019-03-28 2020-10-09 北京沃东天骏信息技术有限公司 一种姿态识别方法、装置及其存储介质
CN110677585A (zh) * 2019-09-30 2020-01-10 Oppo广东移动通信有限公司 目标检测框的输出方法、装置、终端及存储介质
CN111417028A (zh) * 2020-03-13 2020-07-14 腾讯科技（深圳）有限公司 信息处理方法、装置、存储介质及电子设备
CN111897430A (zh) * 2020-07-30 2020-11-06 深圳创维－Rgb电子有限公司 应用的控制方法、显示终端及计算机可读存储介质
CN112543359A (zh) * 2020-11-12 2021-03-23 海信视像科技股份有限公司 一种显示设备及自动配置视频参数的方法
CN112473121A (zh) * 2020-11-13 2021-03-12 海信视像科技股份有限公司 一种显示设备及基于肢体识别的躲避球显示方法

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115065844A (zh) * 2022-05-24 2022-09-16 北京跳悦智能科技有限公司 一种主播肢体动作节奏的自适应调整方法
CN115065844B (zh) * 2022-05-24 2023-09-12 北京跳悦智能科技有限公司 一种主播肢体动作节奏的自适应调整方法
CN115955603A (zh) * 2022-12-06 2023-04-11 广州紫为云科技有限公司 一种基于智慧屏的体感交互的智能摄像头装置和实现方法
CN115955603B (zh) * 2022-12-06 2024-05-03 广州紫为云科技有限公司 一种基于智慧屏的体感交互的智能摄像头装置和实现方法
CN116824475A (zh) * 2022-12-09 2023-09-29 慧之安信息技术股份有限公司 一种基于边缘计算的飞机场智能化安全识别方法和系统
CN116129524A (zh) * 2023-01-04 2023-05-16 长沙观谱红外科技有限公司 一种基于红外图像的姿态自动识别系统和方法
CN116489453A (zh) * 2023-03-31 2023-07-25 海信视像科技股份有限公司 显示设备、终端设备及连屏动作评分显示方法
CN118097027A (zh) * 2024-04-19 2024-05-28 杭州欣禾圣世科技有限公司 一种基于进化计算的多视频3d人体动作捕捉方法及系统
CN119166089A (zh) * 2024-10-11 2024-12-20 广东魅视科技股份有限公司 一种控制分布式显示设备的方法、装置及可读介质

Also Published As

Publication number Publication date
CN116114250B (zh) 2026-02-13
CN116114250A (zh) 2023-05-12

Similar Documents

Publication Publication Date Title
US12284444B2 (en) Display apparatus and camera tracking method
CN116114250B (zh) 显示设备、人体姿态检测方法及应用
CN113655887B (zh) 一种虚拟现实设备及静态录屏方法
CN112866772B (zh) 一种显示设备及声像人物定位追踪方法
CN113596552B (zh) 显示设备及信息显示方法
US12028617B2 (en) Display apparatus and processing method for display apparatus with camera
WO2022001407A1 (zh) 一种摄像头的控制方法及显示设备
WO2020108261A1 (zh) 拍摄方法及终端
CN114637439B (zh) 显示设备和手势轨迹识别方法
WO2022037229A1 (zh) 人像定位方法及显示设备
CN112862859A (zh) 一种人脸特征值创建方法、人物锁定追踪方法及显示设备
CN112672062B (zh) 一种显示设备及人像定位方法
CN117918057A (zh) 显示设备及设备控制方法
CN117980873A (zh) 一种显示设备及其控制方法
CN115623243A (zh) 显示设备、终端设备及动作跟练方法
CN112473121B (zh) 一种显示设备及基于肢体识别的躲避球显示方法
CN113824870A (zh) 显示设备及摄像头角度调整方法
CN114327058A (zh) 显示设备
US12455631B2 (en) Display apparatus
CN116506678B (zh) 一种显示设备及融合虚拟人物的媒资播放方法
CN116489453B (zh) 显示设备、终端设备及连屏动作评分显示方法
CN112423067B (zh) 视频的播放方法、装置、遥控设备及存储介质
CN116126261B (zh) 一种智能设备及目标跟踪显示的方法
CN121501428A (zh) 一种显示设备及跟练模式的控制方法
CN117156208A (zh) 终端设备、显示设备和焦距调节方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21890790

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21890790

Country of ref document: EP

Kind code of ref document: A1