CN114827660A - AI body-building system based on set-top box and implementation method - Google Patents

AI body-building system based on set-top box and implementation method Download PDF

Info

Publication number
CN114827660A
CN114827660A CN202210237125.0A CN202210237125A CN114827660A CN 114827660 A CN114827660 A CN 114827660A CN 202210237125 A CN202210237125 A CN 202210237125A CN 114827660 A CN114827660 A CN 114827660A
Authority
CN
China
Prior art keywords
image
server platform
top box
fitness
images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210237125.0A
Other languages
Chinese (zh)
Inventor
文太益
周灯杨
李雨菲
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wasu Media & Network Co ltd
Original Assignee
Wasu Media & Network Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wasu Media & Network Co ltd filed Critical Wasu Media & Network Co ltd
Priority to CN202210237125.0A priority Critical patent/CN114827660A/en
Publication of CN114827660A publication Critical patent/CN114827660A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • H04N21/2343Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
    • H04N21/234336Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by media transcoding, e.g. video is transformed into a slideshow of still pictures or audio is converted into text
    • AHUMAN NECESSITIES
    • A63SPORTS; GAMES; AMUSEMENTS
    • A63BAPPARATUS FOR PHYSICAL TRAINING, GYMNASTICS, SWIMMING, CLIMBING, OR FENCING; BALL GAMES; TRAINING EQUIPMENT
    • A63B71/00Games or sports accessories not covered in groups A63B1/00 - A63B69/00
    • A63B71/06Indicating or scoring devices for games or players, or for other sports activities
    • A63B71/0619Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills
    • A63B71/0622Visual, audio or audio-visual systems for entertaining, instructing or motivating the user
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L69/00Network arrangements, protocols or services independent of the application payload and not provided for in the other groups of this subclass
    • H04L69/16Implementation or adaptation of Internet protocol [IP], of transmission control protocol [TCP] or of user datagram protocol [UDP]
    • H04L69/164Adaptation or special uses of UDP protocol
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/234Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41Structure of client; Structure of client peripherals
    • H04N21/422Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/4223Cameras

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Computer Security & Cryptography (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Closed-Circuit Television Systems (AREA)

Abstract

The invention relates to an AI body-building system based on a set-top box and a realization method thereof, wherein the system comprises the set-top box, a server platform and a display device; the set top box is in communication connection with the server platform and is also connected with the display equipment; the set-top box comprises a shooting part, wherein the shooting part is used for acquiring an image and uploading the image to the server platform; the display device is used for displaying the shot image and displaying a set area in the image; the server platform is used for acquiring a set area in the image, and the server platform is used for acquiring the set area on the image; the set top box end collects images, and the server platform processes the images to obtain Google nodes in the images, so that the set top box with low computing power can realize AI fitness business.

Description

AI body-building system based on set-top box and implementation method
Technical Field
The invention relates to the field of set top boxes, in particular to an AI body-building system based on a set top box and an implementation method.
Background
With the improvement of living standard, people pay more attention to the physical health of individuals. For most working people, there is not much exercise time in the working day, so people can choose to exercise to meet the exercise amount requirement of the body. In the process of body building, the limbs are usually required to complete some specified actions, and for the specified actions, the external environment is required to supervise, including that an intelligent terminal acquires images to supervise, manual supervision is performed by a coach, and the like.
In the field of fitness, intelligent terminals comprise an internet OTT intelligent set top box and a mobile terminal, a terminal camera is mostly adopted to collect video images, an embedded neural Network Processor (NPU) module of a terminal chip is utilized to complete logic reasoning based on artificial intelligence, and results are output. The embedded neural Network Processor (NPU) adopts a data-driven parallel computing architecture and is good at processing massive multimedia data such as videos and images. For a terminal chip containing an NPU module, such as an entry-level Amlogic S905D 3 chip and the like, the requirement of fitness business can be met; on the other hand, the terminal chip has high price, and the price ratio is extremely low when the terminal chip is applied to a set top box, so that the terminal chip is not suitable for popularization.
However, the terminal chip of the existing set-top box does not include an NPU module, but directly utilizes a GPU or a CPU to perform AI logical reasoning, the operation speed is low, and the requirement of high real-time property of AI body building is difficult to meet. Such as Hi3798MV200H, GK6323V100A and MSO9385AD adopted by 4K set-top box main chips in the broadcasting and television market; hi3798MV310, AML S905L3 and MSO9385AK adopted by a 4K set-top box main chip in the IPTV market; the chip is characterized in that the chip is an Ampogic S905 series chip and Hi3798MV310 adopted by a 4K set top box main chip in the OTT market. The current set-top box chip configuration is difficult to realize and popularize AI body-building services. Therefore, a system and method for performing AI fitness by platform services is needed.
Disclosure of Invention
The invention aims to overcome the defects in the prior art and provides an AI body-building system based on a set top box and an implementation method.
In order to solve the problems, the invention adopts the following technical scheme:
an AI body-building system based on a set-top box comprises the set-top box, a server platform and a display device; the set-top box is in communication connection with the server platform and is also connected with the display equipment; the set-top box comprises a shooting part, wherein the shooting part is used for acquiring an image and uploading the image to the server platform; the display device is used for displaying the shot image and displaying a set area in the image; the server platform is used for acquiring a set area in the image, and the server platform is used for acquiring the set area on the image.
Further, a camera shooting part in the set top box is a USB camera.
Further, the server platform comprises an image recognition module; the image recognition module employs a GPU cluster.
An AI body-building implementation method based on a set top box comprises the following steps:
step 1: the method comprises the steps that images are collected in real time through a camera shooting part in a set top box, and collected images and image identification requests are transmitted to a server platform;
and 2, step: the server platform receives the image identification requests and judges whether the number of the acquired image identification requests exceeds a set threshold value or not; if the video transcoding service exceeds the set threshold, closing the Docker instance of the video transcoding service; otherwise, starting a Docker instance of the video transcoding service;
and step 3: the server platform identifies skeleton nodes in the image through the GPU cluster and returns skeleton node data;
and 4, step 4: the set top box receives the skeleton node data returned by the server platform and displays the skeleton nodes on the display equipment;
and 5: the display device also displays the images collected by the camera part or the reference images of the body-building exercise.
Furthermore, in the step 1, the image acquired by the image pickup part is output in a YUV format and is directly uploaded through a QUIC protocol based on UDP.
Further, the set-top box is connected with the server platform through a network; the bandwidth of the network is above 30M.
Further, the bone node data returned in step 3 includes names of bone nodes and coordinates and areas of the bone nodes; the coordinates of the skeleton nodes are the central coordinates of the skeleton nodes obtained by identification; the area of the bone node is a preset area size, wherein the area of the bone node corresponds to the name of the bone node.
Further, the data returned by the server platform in step 4 further includes evaluation of the degree of the motion criterion, a score of the completed motion, and a motion count, but does not include the received image data.
Further, in step 5, the display device can switch the displayed images through external manipulation, including the image acquired by the camera portion and the reference image for the fitness exercise.
The invention has the beneficial effects that:
acquiring an image through a set top box end, processing the image through a server platform, and acquiring skeleton nodes in the image, so that the set top box without an NPU module can realize AI fitness service;
by converting the functions of the server platform, including video transcoding and AI body-building services, the utilization efficiency of computing power of the server platform is improved, and resource waste is avoided;
through a QUIC protocol based on UDP, the collected video images are directly uploaded, the transmission efficiency is improved, the end-to-end time delay is reduced, in addition, the skeleton node data is returned through a server platform, the data transmission quantity is reduced, and the transmission efficiency is further improved.
Drawings
FIG. 1 is a schematic structural connection diagram according to a first embodiment of the present invention;
fig. 2 is a schematic data flow diagram according to a first embodiment of the invention.
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in practical implementation, and the type, quantity and proportion of the components in practical implementation can be changed freely, and the layout of the components can be more complicated.
The first embodiment is as follows:
as shown in fig. 1 and 2, an AI fitness system based on a set-top box includes a set-top box, a server platform and a display device; the set-top box is in communication connection with the server platform and is also connected with the display equipment; the set-top box comprises a shooting part, wherein the shooting part is used for acquiring an image and uploading the image to the server platform; the display device is used for displaying the shot image and displaying a set area in the image, wherein the set area is a bone node in the example; the server platform is used for acquiring a set area in the image, wherein the server platform acquires the set area on the image through the existing image recognition technology.
The camera part in the set top box is a USB camera, the resolution of the camera is 1280 × 960, 720 × 480, 640 × 480 or 320 × 240, and the like, in the example, the resolution of the camera is 320 × 240; the purpose of the low resolution image is to increase the speed of the video stream transmitted from the set-top box to the server platform.
The server platform comprises an image identification module, the image identification module adopts a GPU cluster in the embodiment, the GPU cluster can be used for video transcoding and AI fitness services, and the AI fitness services comprise identification of skeleton nodes in uploaded images.
An AI body-building implementation method based on a set top box comprises the following steps:
step 1: the method comprises the steps that images are collected in real time through a camera shooting part in a set top box, and collected images and image identification requests are transmitted to a server platform;
step 2: the server platform receives the image identification requests and judges whether the number of the acquired image identification requests exceeds a set threshold value or not; if the video transcoding service exceeds the set threshold, closing the Docker instance of the video transcoding service; otherwise, starting a Docker instance of the video transcoding service;
and step 3: the server platform identifies skeleton nodes in the image through the GPU cluster and returns skeleton node data;
and 4, step 4: the set top box receives the skeleton node data returned by the server platform and displays the skeleton nodes on the display equipment;
and 5: the display device also displays the images collected by the camera part or the reference images of the body-building exercise.
In the step 1, the image acquired by the camera part is output in a YUV format and directly uploaded through a QUIC protocol based on UDP, and it should be noted that, in the set-top box part, the image is not compressed, and the overall speed is increased. The set-top box is connected to the server platform via a network, which in this example requires a bandwidth of more than 30M.
The image recognition request in the step 2 is realized based on the K8s architecture.
The skeleton node data returned in the step 3 comprises the names of skeleton nodes and the coordinates and the areas of the skeleton nodes; wherein the names of the bone nodes comprise knee joints, elbow joints, necks and the like; the coordinates of the bone nodes are the central coordinates of the bone nodes obtained by identification, and in the embodiment, the central coordinates of the bone node areas are equal; the area of the skeleton node is a preset area size, wherein the area of the skeleton node corresponds to the name of the skeleton node, and the center coordinate of the area of the skeleton node is the coordinate of the skeleton node. It should be noted that in this example, the server platform does not return the received image data, so as to reduce the size of the transmission file and increase the transmission speed. Wherein the method for identifying the bone nodes of the image comprises training an image identification model and the like. In this example, the output speed of the model is also increased by improving the hardware performance and optimizing the training data. The data returned by the server platform also includes the evaluation of the degree of the motion criteria, the completed action score, and the action count, but does not include the received image data.
In the step 5, the image type displayed by the display device can be switched through external control, and the image type comprises the image collected by the camera part and the reference image for body building and training. If the image collected by the camera shooting part is displayed, the body-building personnel can conveniently and directly observe the self action; if the reference image of the fitness exercise is displayed, the fitness personnel can correct the self action conveniently. In this example, the display device may also display an action score, which is obtained by the set-top box or server platform, and the action score is obtained by comparing the set bone node coordinates in the reference image data with the bone node data obtained from the image.
In step 5, in order to ensure that the returned bone node data can correspond to the image data displayed on the display, the displayed image data is also adjusted according to the time corresponding to the bone node. The returned skeleton node data also comprises time nodes, when the display displays images, the time node data in the skeleton node data can be preferentially read, and the images are displayed according to the read time node data; in this example, if the time difference between the read bone node data and the displayed image data exceeds a set threshold, the displayed image data is calibrated to ensure the synchronization performance of the displayed image and the bone node data; the set threshold value of the time difference is 0.5 s.
In the implementation process, the delay of returning the skeleton node data can be controlled within 150ms, and the real-time requirement of AI fitness is met; acquiring an image through a set top box end, and processing the image through a server platform to obtain skeleton nodes in the image, so that the set top box without the NPU can realize AI fitness service; by converting the functions of the server platform, including video transcoding and AI body-building services, the utilization efficiency of computing power of the server platform is improved, and resource waste is avoided; through a QUIC protocol based on UDP, the collected video images are directly uploaded, the transmission efficiency is improved, the end-to-end time delay is reduced, in addition, the skeleton node data is returned through a server platform, the data transmission quantity is reduced, and the transmission efficiency is further improved.
The above description is only one specific example of the present invention and should not be construed as limiting the invention in any way. It will be apparent to persons skilled in the relevant art(s) that, having the benefit of this disclosure and its principles, various modifications and changes in form and detail can be made without departing from the principles and structures of the invention, which are, however, encompassed by the appended claims.

Claims (9)

1. An AI body-building system based on a set-top box is characterized by comprising the set-top box, a server platform and a display device; the set-top box is in communication connection with the server platform and is also connected with the display equipment; the set-top box comprises a shooting part, wherein the shooting part is used for acquiring an image and uploading the image to the server platform; the display device is used for displaying the shot image and displaying a set area in the image; the server platform is used for acquiring a set area in the image, and the server platform is used for acquiring the set area on the image.
2. The set-top-box-based AI fitness system of claim 1, wherein the camera in the set-top-box is a USB camera.
3. The set-top-box-based AI fitness system of claim 1, wherein the server platform comprises an image recognition module; the image recognition module employs a GPU cluster.
4. An AI body-building implementation method based on a set top box is characterized by comprising the following steps:
step 1: the method comprises the steps that images are collected in real time through a camera shooting part in a set top box, and collected images and image identification requests are transmitted to a server platform;
step 2: the server platform receives the image identification requests and judges whether the number of the acquired image identification requests exceeds a set threshold value or not; if the threshold value is exceeded, closing a Docker instance of the video transcoding service; otherwise, starting a Docker instance of the video transcoding service;
and step 3: the server platform identifies skeleton nodes in the image through the GPU cluster and returns skeleton node data;
and 4, step 4: the set top box receives the skeleton node data returned by the server platform and displays the skeleton nodes on the display equipment;
and 5: the display device also displays the images collected by the camera part or the reference images of the body-building exercise.
5. The AI body-building implementation method of claim 4, wherein in step 1, the image captured by the camera is outputted in YUV format and uploaded directly via QUIC protocol based on UDP.
6. The AI fitness implementing method of claim 5, wherein the set-top box is connected to the server platform via a network; the bandwidth of the network is above 30M.
7. The AI fitness implementation method of claim 4, wherein the skeletal node data returned in step 3 comprises the name of the skeletal node and the coordinates and area of the skeletal node; the coordinates of the skeleton nodes are the central coordinates of the skeleton nodes obtained by identification; the area of the bone node is a preset area size, wherein the area of the bone node corresponds to the name of the bone node.
8. The AI fitness implementation method of claim 7, wherein the data returned by the server platform in step 4 further includes an evaluation of the degree of exercise criteria, a score of the completed action, and a count of the action, but does not include the received image data.
9. The AI fitness implementing method based on the set-top box of claim 4, wherein in the step 5, the display device can switch the displayed images through external manipulation, including the image collected by the camera and the reference image of the fitness exercise.
CN202210237125.0A 2022-03-11 2022-03-11 AI body-building system based on set-top box and implementation method Pending CN114827660A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210237125.0A CN114827660A (en) 2022-03-11 2022-03-11 AI body-building system based on set-top box and implementation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210237125.0A CN114827660A (en) 2022-03-11 2022-03-11 AI body-building system based on set-top box and implementation method

Publications (1)

Publication Number Publication Date
CN114827660A true CN114827660A (en) 2022-07-29

Family

ID=82528877

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210237125.0A Pending CN114827660A (en) 2022-03-11 2022-03-11 AI body-building system based on set-top box and implementation method

Country Status (1)

Country Link
CN (1) CN114827660A (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105025200A (en) * 2015-08-06 2015-11-04 成都市斯达鑫辉视讯科技有限公司 Method for supervising user by set top box
CN105681440A (en) * 2016-01-29 2016-06-15 浪潮软件集团有限公司 Method and equipment for counting exercise amount
CN110478883A (en) * 2019-08-21 2019-11-22 南京信息工程大学 A kind of body-building movement teaching and correction system and method
CN110478862A (en) * 2019-05-07 2019-11-22 深圳市云康创新网络科技有限公司 A kind of exercise guide system and its guidance method
CN111111111A (en) * 2020-01-14 2020-05-08 广东技术师范大学 Real-time fitness monitoring system and method
CN111931725A (en) * 2020-09-23 2020-11-13 北京无垠创新科技有限责任公司 Human body action recognition method, device and storage medium
CN113144540A (en) * 2021-03-12 2021-07-23 华南理工大学 Intelligent safe fitness guidance device, system and method
CN113516064A (en) * 2021-07-02 2021-10-19 深圳市悦动天下科技有限公司 Method, device, equipment and storage medium for judging sports motion
CN113873279A (en) * 2021-09-27 2021-12-31 广州中工水务信息科技有限公司 Video data decoding method, system and storage medium
CN114091511A (en) * 2021-09-22 2022-02-25 广东工业大学 Fitness action scoring method, system and device based on space-time information

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105025200A (en) * 2015-08-06 2015-11-04 成都市斯达鑫辉视讯科技有限公司 Method for supervising user by set top box
CN105681440A (en) * 2016-01-29 2016-06-15 浪潮软件集团有限公司 Method and equipment for counting exercise amount
CN110478862A (en) * 2019-05-07 2019-11-22 深圳市云康创新网络科技有限公司 A kind of exercise guide system and its guidance method
CN110478883A (en) * 2019-08-21 2019-11-22 南京信息工程大学 A kind of body-building movement teaching and correction system and method
CN111111111A (en) * 2020-01-14 2020-05-08 广东技术师范大学 Real-time fitness monitoring system and method
CN111931725A (en) * 2020-09-23 2020-11-13 北京无垠创新科技有限责任公司 Human body action recognition method, device and storage medium
CN113144540A (en) * 2021-03-12 2021-07-23 华南理工大学 Intelligent safe fitness guidance device, system and method
CN113516064A (en) * 2021-07-02 2021-10-19 深圳市悦动天下科技有限公司 Method, device, equipment and storage medium for judging sports motion
CN114091511A (en) * 2021-09-22 2022-02-25 广东工业大学 Fitness action scoring method, system and device based on space-time information
CN113873279A (en) * 2021-09-27 2021-12-31 广州中工水务信息科技有限公司 Video data decoding method, system and storage medium

Similar Documents

Publication Publication Date Title
EP3876140B1 (en) Method and apparatus for recognizing postures of multiple persons, electronic device, and storage medium
US20220329880A1 (en) Video stream processing method and apparatus, device, and medium
KR102161034B1 (en) System for providing exercise lecture and method for providing exercise lecture using the same
CN107945255A (en) A kind of virtual actor's facial expression driving method and system
CN112601022B (en) On-site monitoring system and method based on network camera
CN110087081A (en) Method for video coding, device, server and storage medium
CN110478862A (en) A kind of exercise guide system and its guidance method
CN109961039A (en) A kind of individual's goal video method for catching and system
CN115631460A (en) Commodity state identification method based on improved YOLOv5 target detection
CN114827660A (en) AI body-building system based on set-top box and implementation method
CN109413152A (en) Image processing method, device, storage medium and electronic equipment
CN112354185A (en) Cloud game control system and cloud game control method
CN112183430A (en) Sign language identification method and device based on double neural network
CN112069979A (en) Real-time action recognition man-machine interaction system
CN110213611A (en) A kind of ball competition field camera shooting implementation method based on artificial intelligence Visual identification technology
CN114495169A (en) Training data processing method, device and equipment for human body posture recognition
CN111523495B (en) End-to-end active human body tracking method in monitoring scene based on deep reinforcement learning
CN113610980A (en) Video conference method, terminal and system based on motion capture and three-dimensional reconstruction
CN113347400A (en) Interesting projection system for children dance match and using method thereof
CN113132754A (en) Motion video clipping method and system based on 5GMEC
CN209865244U (en) Goal video capture device
CN112383788A (en) Live broadcast real-time image extraction system and method based on intelligent AI technology
CN117061792B (en) Cloud video collaborative rendering method and device and computer equipment
CN112295211A (en) Stage performance virtual entertainment practical training system and method
CN117194970A (en) Motorized platform model updating method and system for data distillation

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination