WO2021103610A1

WO2021103610A1 - Display object control method and apparatus, electronic device and storage medium

Info

Publication number: WO2021103610A1
Application number: PCT/CN2020/104483
Authority: WO
Inventors: 孙贺然
Original assignee: 北京市商汤科技开发有限公司
Priority date: 2019-11-28
Filing date: 2020-07-24
Publication date: 2021-06-03
Also published as: TWI758837B; KR20210075188A; TW202121250A; CN110968239A; CN110968239B; JP2022515317A

Abstract

Provided are a display object control method and apparatus, an electronic device, and a storage medium. The method comprises: displaying an acquired video picture in a display screen; where it is detected that the video picture comprises a target object, displaying, in the display screen, a display object associated with the target object; and controlling, on the basis of an accumulated detection result of the target object in the video picture, the display object in the display screen to change display state.

Description

Display object control method and device, electronic equipment and storage medium

Cross-references to related applications

This disclosure claims the priority of a Chinese patent application filed on November 28, 2019, with an application number of 201911190043.X and an invention title of "a method, device, equipment, and storage medium for displaying objects". The Chinese patent The entire contents of the application disclosure are incorporated herein by reference.

Technical field

The present disclosure relates to the field of computer technology, and in particular to a method and device for controlling display objects, electronic equipment and storage media.

Background technique

When events are held in some venues or areas, statistics are usually made on the information of the attendees. The information statistics of the attendees are often done manually by specialized staff, such as signing the sign-in form, entering identity information, etc. The operation is cumbersome and the operation process is simplistic, which is not conducive to the interaction with the attendees.

Summary of the invention

In view of this, the present disclosure provides at least a solution for controlling the display object.

In a first aspect, the present disclosure provides a method for controlling a display object, including: displaying the acquired video picture on a display screen; in a case where it is detected that the video picture includes a target object, displaying on the display screen A display object associated with the target object; based on a cumulative detection result of the target object in the video screen, controlling the display object in the display screen to change a display state.

The above provides an automatic detection scheme for the target object, which can detect whether the target object is included in the video image based on the video image displayed on the display screen, and display the target object when the target object is detected in the video image. The associated display object, and based on the cumulative detection result of the target object in the video screen, controls the display object to change the display state. As the target object changes state information (such as gestures, smiles, etc.), the cumulative detection result can also be changed accordingly, and the display object can be controlled to change the display state, thereby enriching the display effect and enhancing the interaction with the target object.

In a second aspect, the present disclosure provides a display object control device, including: a first display module for displaying the acquired video picture on a display screen; a second display module for displaying the video picture in the If the target object is included, the display object associated with the target object is displayed on the display screen; the control module is configured to control the display screen based on the cumulative detection result of the target object in the video screen The display object in changes the display state.

In a third aspect, the present disclosure provides an electronic device including: a processor; and a memory, wherein the memory stores machine-readable instructions executable by the processor, and the machine-readable instructions are executed by the processor At the time, the processor is made to implement the control method of the display object described in the first aspect.

In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored on the computer-readable storage medium, wherein when the computer program is run by a processor, the processor realizes the above-mentioned first aspect The control method of the display object.

In order to make the above objectives, features and advantages of the present disclosure more comprehensible, preferred embodiments are described below in detail with accompanying drawings.

Description of the drawings

In order to explain the embodiments of the present disclosure more clearly, the following will briefly introduce the drawings that need to be used in the embodiments. The drawings here are incorporated into the specification and constitute a part of the specification. These drawings show The embodiments conforming to the present disclosure are presented, and together with the description are used to describe the embodiments of the present disclosure. It should be understood that the following drawings only show certain embodiments of the present disclosure, and therefore should not be regarded as limiting the scope. For those of ordinary skill in the art, they can also Obtain other related drawings based on these drawings.

FIG. 1 shows a schematic flowchart of a method for controlling a display object provided by an embodiment of the present disclosure;

FIG. 2 shows a schematic diagram of an interface of a first display area provided by an embodiment of the present disclosure;

FIG. 3 shows a schematic diagram of a special effect display provided by an embodiment of the present disclosure;

FIG. 4 shows a schematic diagram of a second display object display provided by an embodiment of the present disclosure;

FIG. 5 shows a schematic diagram of a method for determining a movement route of a target object provided by an embodiment of the present disclosure;

FIG. 6 shows a schematic diagram of the distribution of various display areas in a display screen provided by an embodiment of the present disclosure;

FIG. 7 shows a schematic diagram of the distribution of various display areas in another display screen provided by an embodiment of the present disclosure;

FIG. 8 shows a schematic structural diagram of a display object control device provided by an embodiment of the present disclosure;

FIG. 9 shows a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure.

Detailed ways

In order to make the purpose and advantages of the embodiments of the present disclosure clearer, the embodiments of the present disclosure will be described clearly and completely in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments. The components of the embodiments of the present disclosure generally described and illustrated in the drawings herein may be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the accompanying drawings is not intended to limit the scope of the claimed present disclosure, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without creative work shall fall within the protection scope of the present disclosure.

The terms used in the present disclosure are only for the purpose of describing specific embodiments, and are not intended to limit the present disclosure. The singular forms used in the present disclosure such as "a", "said", "the", etc. are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and/or" as used herein refers to and includes any one or all possible combinations of one or more associated listed items.

It should be understood that, although the terms first, second, third, etc. may be used in the present disclosure to describe various information, these information should not be limited by these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present disclosure, the first information may also be referred to as second information, and similarly, the second information may also be referred to as first information. Depending on the context, the word "if" as used herein can be interpreted as "when" or "when" or "in response to".

First, the application scenarios applicable to the embodiments of the present disclosure are introduced. The present disclosure can be applied to electronic equipment with data processing capabilities. The electronic equipment can be equipped with a display device for displaying data processing results, or an external display device for displaying data processing results. The connection method is not limited to wired connection, and/ Or wireless connection. The electronic device may also be connected to at least one camera device, and the connection mode is not limited to wired connection and/or wireless connection. The electronic equipment and the camera connected to it can be deployed in the same location area. After the camera takes the video, it is transmitted to the electronic equipment for processing. After the processing is completed, the electronic equipment can be externally connected through the display device configured by the electronic equipment or the electronic equipment. Display device (such as a display screen) for display. Among them, the electronic device may be, for example, a mobile phone, a tablet computer, a smart TV, a computer, and the like.

In some embodiments, the target object may be all or part of the users appearing in the video screen, or users appearing in the video screen and conforming to a certain set attribute, or a specific object set in advance. Different target objects can be determined according to different application scenarios.

In some embodiments, the cumulative detection result is the cumulative value of the detection results of a certain attribute of the target object or a certain posture of the target object at different times. Specific attributes or postures can be set according to different application scenarios, and the detection results are not limited to the above two cases.

In order to facilitate the understanding of the embodiments of the present disclosure, first, the method for controlling display objects provided by the present disclosure will be introduced in detail.

Referring to FIG. 1, a method for controlling a display object provided by an embodiment of the present disclosure includes steps S101 to S103.

S101. Display the acquired video picture on the display screen.

Among them, the video picture displayed on the display screen may be obtained by real-time shooting by a camera device, which may be a camera built into the electronic device or an external camera device. If the camera device is an external device of the electronic device, the camera device and the display screen can be deployed in the same location area, for example, can be deployed in the same exhibition or in the same room. Considering that the video screen shot by the camera device is displayed on the display screen, in order to make it easier for users to watch their posture (such as facial posture and/or body posture), the camera device can be placed on the same vertical plane as the display screen. . Alternatively, the position where the user can capture the complete posture of the user when the user views the display screen can be used as the deployment position of the camera device.

S102: In a case where it is detected that the target object is included in the video picture, display the display object associated with the target object on the display screen.

There can be multiple display objects associated with a target object, and different target objects can be associated with different display objects. The specifics will be described in detail in the following embodiments, and the description will not be expanded here.

S103: Based on the cumulative detection result of the target object in the video screen, control the display object in the display screen to change the display state.

In the embodiments of the present disclosure, it is possible to detect whether the target object is included in the video picture based on the video picture displayed on the display screen, and when the target object is detected in the video picture, display the display object associated with the target object, and based on the comparison The cumulative detection results of the target object in the video screen, and control the display object to change the display state. As the target object changes state information (such as gestures, smiles, etc.), the cumulative detection result can also be changed accordingly, and the display object can be controlled to change the display state, thereby enriching the display effect and enhancing the interaction with the target object.

The target object in step 102 may be all objects that appear in the image acquisition range, or may be part of the object that can obtain a complete face in the image acquisition range, or may be within the image acquisition range that meets the preset attribute characteristics Object. Among them, the preset attribute characteristics can be, for example, gender, hair length, clothes color, and so on. If the target object is an object that meets the preset attribute characteristics, after acquiring the video image, the attribute characteristics of each object in the video image can be identified first, and then the object that meets the preset attribute characteristics is determined as the target object.

In another possible implementation manner, the target object may also be a specific object (there may be one or more), where the specific object may be, for example, a person who is scheduled to participate in an exhibition. In the case where the target object is a specific object, the biological characteristics of the specific object may be stored in advance, for example, facial characteristics, gait characteristics, pedestrian re-identification (Person Re-identification, REID) characteristics, and so on. Detecting whether the target object is included in the video frame may include: detecting the biological characteristics of each object in the video frame, matching the detected biological characteristics with the biological characteristics of the specific object stored in advance, and determining the successfully matched object as the target object.

In a possible implementation manner, in a case where it is detected that the target object is included in the video frame, displaying the display object associated with the target object may include one or more of the following six situations.

Case 1: In the case where the target object is detected in the video frame, the first display object in the initial display state can be displayed in the first display area of the display screen, where the initial display state is used to prompt the target object to make a target gesture.

Target gestures include but are not limited to postures and facial expressions, such as hand gestures, smiling expressions, etc.

Among them, the acquired video images can also be displayed in the first display area of the display screen. When displaying the first display object in the initial display state, the key points of each target object in the video images can be detected first, and then based on the key points The first display object in the initial display state is displayed in the first display area.

Exemplarily, FIG. 2 is a schematic diagram of the interface of the first display area. As shown in FIG. 2, the first display area displays a video picture taken by the camera. The video picture includes multiple target objects, and the head of each target object Special effects of flowers are displayed in this position. The displayed flower is the first display object in the initial display state. There is a prompt “smile becomes VIP” around the flower to remind the user to smile (smile is the target posture); in another example, the prompts around the flower are displayed After that, the prompt language can also be voice broadcast to give voice prompts to the target object.

In some embodiments of the present disclosure, it can also be applied to classroom scenes. The first display area may display the classroom images captured by the camera in real time, and each student in the captured classroom images is a target object. In each student's head position, there can be a palm special effect, and the palm special effect is the first display object. The palm special effect in the initial display state can be a palm with a dashed border, with a prompt "raise hand to ask questions" around the palm to remind students to raise their hands for interaction. In a possible example, after the prompt words are displayed around the palm of the hand, the displayed prompt words can also be voice-broadcasted to give voice prompts to the students.

In specific implementation, based on the cumulative detection results of the target object in the video screen, controlling the display object in the display screen to change the display state may include: detecting the target posture made by the target object in the video frame, and making the cumulative target posture based on the target object The duration or cumulative number of times controls the change of the display state of the first display object in the first display area.

In a possible implementation, the controlling the first display object in the first display area to change the display state based on the cumulative duration or cumulative number of times the target object makes the target gesture may include: when the cumulative duration or cumulative number of times has not reached In the case of the set value, the first display object is controlled to change the display state to the intermediate display state; wherein, the intermediate display state changes with the cumulative value of the cumulative duration or cumulative times to indicate the cumulative value.

As shown in Figure 2, if the first display object in the initial display state is a flower special effect, when the first display object in the first display area is controlled to change the display state, the color of the flower can be changed according to the smile duration of the target user, or Light up the square in the progress bar below the flower. For example, if a square in the progress bar means a smile for 2 seconds, when the target object smiles for 10 seconds, light up 5 squares in the progress bar under the flower from bottom to top. In addition, the color of the squares can be changed according to the number of illuminated squares. For example, when only 5 squares are lit, the color of the squares can be blue; as the number of lighted squares increases, the color of the squares can be gradually changed; when all the squares are lighted, the squares can be red.

As shown in Figure 3, in the classroom application scenario, the number of hands raised by the student (ie the target object) can be detected, and the displayed palm special effects can be changed according to the number of hands raised by the student. If the student raises their hands 4 times, they can light up the palm special effect and add a "×4" sign behind the palm special effect. If the student raises his hand for the first time, he can show only one palm special effect at the top of his head, and add the “×1” sign behind the displayed palm special effect.

In another possible implementation manner, the controlling the first display object in the first display area to change the display state based on the cumulative duration or cumulative number of the target gestures made by the target object may include: When the number of times reaches the set value, the first display object is controlled to change the display state to the target display state to indicate that the target posture made by the target object meets the set condition.

In the above-mentioned classroom application scenario, when it is detected that the number of times the student raises their hands reaches the set number of times, the first display object may include not only the palm special effect, but also the star special effect; the target display state of the first display object may be, for example, the star special effect The five stars of will light up in turn to indicate that the number of times the student has raised their hands has reached the set number of times; at the same time, a sign indicating the number of times the hand has been raised is added after the palm special effect.

In case 1, by displaying the first display object in the initial display state, the target object can be prompted to make a target posture, thereby increasing the interaction with the target object, and by controlling the first display object to change the display state, it becomes the first display object Provides more ways to display.

Case 2: In the case where the target object is detected in the video frame, the attribute feature of the target object in the video frame can be detected, and the second display object corresponding to the attribute feature can be displayed in the first display area of the display screen.

Wherein, the second display object includes at least one of the following information: the identity of the target object, age value, smile value, charm value, viewing duration, duration of different emotions, duration of attention. In different application scenarios, the type of the second display object corresponding to the attribute feature may be different, which can be specifically set according to actual conditions.

Exemplarily, when some embodiments of the present disclosure are applied in a classroom scene, the target object may be a student in the classroom, and the corresponding second display object may include at least one of the following information: duration of class, duration of attention , Positive emotion duration, negative emotion duration. Among them, positive emotions can be happiness, excitement, excitement, etc., and negative emotions can be fear, sadness, etc.; class duration can be the length of time between when the electronic device starts to acquire the video image captured by the camera and the current moment; The attention time period may be the time period during which the camera device can collect the complete facial features of the student. In some embodiments, the length of attention can be determined in the following manner: by detecting the head position of the student in each frame of the video picture, the offset angle between the head position of the student and the preset position (for example, a blackboard) is determined; When the offset angle is within the preset angle range, it is determined that the student is paying attention to the blackboard at the moment; the attention duration is determined by counting the number of video frames in which the student is paying attention to the blackboard.

In some embodiments, the positive emotion duration and negative emotion duration of the students can be detected by the following method: input the captured video images into the pre-trained emotion recognition model to predict the corresponding value of each student in the video image. Emotions; determine whether the emotion is a positive emotion or a negative emotion based on the predicted emotions; count the duration of the video images where the student’s emotions belong to the positive emotion and the video images where the student’s emotions belong to the negative emotion as of the current moment The duration of the positive emotion and the duration of the negative emotion corresponding to each student are determined based on the duration of the video images in which each emotion appears.

Some embodiments of the present disclosure may also be applied to public places such as exhibitions or shopping malls. In this case, the second display object may include one or more of identity, age, charm value, viewing time, expression, and smile value.

Among them, the attribute characteristics of the target object may be the biological characteristics of the target object (such as facial characteristics, gait characteristics, REID characteristics, etc.). Exemplarily, the biological characteristics of the target object in the video frame can be extracted first, and then the extracted biological characteristics can be matched with the biological characteristics of the known identities stored in the database, and the identity of the biological characteristics in the database that has been successfully matched The identification is determined as the identification identification of the target object; if the matching is unsuccessful, an identification identification is assigned to the target object, and the identification identification assigned to the target object and its corresponding biological characteristics are stored in the database.

When determining the age of the target object, the video image can be input into a pre-trained neural network model to predict the age of the corresponding target object in the video image. The detection method of the target object's charm value, expression, smile value and other attribute characteristics can be similar to the detection method of the target object's age, which will not be repeated here, but it needs to be explained that the neural network model used to predict different attribute characteristics Different, the training process of different neural network models is also different.

When the attribute characteristics of the target object include multiple characteristics of age, charm value, expression, and smile value, it is also possible to predict multiple characteristics of the target user in the video screen based on the same neural network model, but what needs to be explained is When training the neural network model, the labels of the supervised data need to be multiple feature labels.

The method for determining the viewing time of the target object may be the same as the method for determining the student's attention time in the classroom scene, which will not be repeated here.

Considering that the state of the target object (such as posture, facial expression, etc.) may change all the time, therefore, the attribute characteristics of the target object may also change. For example, if the target object changes facial expression, the expression, smile value, charm value, etc. of the detected target object may change.

In a possible implementation manner, in the case of detecting that the attribute characteristic of the target object changes, it is also possible to control the second display object in the first display area to change the display state according to the change of the attribute characteristic. Exemplarily, the second display object may be as shown in FIG. 4, and a display frame may be displayed around each target object. The display frame may display at least one of the identity, age, charm value, expression, viewing time, and smile value of the target object, and the display position of the display frame may be a fixed position around the target object. For example, the key points of the body of the target object can be detected, and then the display position can be determined according to the key points of the body, and then the display frame corresponding to the target object can be displayed.

In the above solution, by displaying the second display object corresponding to the attribute characteristics of the target object, personalized display of different target objects can be realized, and the display needs of different target objects can be met.

Case 3: When it is detected that the target object is included in the video picture, the sign-in information of the target object may be displayed in the second display area of the display screen.

In related technologies, when the information of attendees needs to be counted when an event is held, it is often necessary to perform manual statistics by specialized staff, such as signing a sign-in form, entering identity information, etc., which is cumbersome and has low sign-in efficiency.

In some embodiments, the target object may be a predetermined attendee. For example, if the application scenario is a meeting, the target object is a predetermined participant.

In a possible implementation manner, the biological characteristics of the target object (such as facial characteristics, gait characteristics, REID characteristics, etc.) can be input in advance. After the video frame is acquired, the biological characteristics of each object in the video frame are detected, the detected biological characteristics are compared with the biological characteristics input in advance, and the objects whose comparison results are successful are determined as the target objects.

The check-in information of the target object may be displayed in the second display area, and the check-in information may include, for example, a photo of the target object and/or the identity information of the target object. After detecting that the target object is included in the video picture, the display state of the sign-in information of the target object in the second display area can be changed.

In some embodiments, the sign-in information may be displayed in the form of information cards in the second display area, and each information card displays a photo of the target object and/or identity information of the target object. The background color of the information card can be changed when the display status of the sign-in information is changed, or the color of the information card can be gray before the display status of the sign-in information is changed, and the display status of the sign-in information can be changed by lighting the information card, etc.

In another possible implementation manner, when the display object in the first display area is controlled to change its display state, and it is detected that the changed display state is the target display state, the display in the second display area may also be marked The sign-in information of the target object.

Exemplarily, if the display object in the first display area is a flower special effect, there may be a prompt "Smile becomes VIP" beside the flower special effect. When the target object is detected to smile, the color of the flower special effect is controlled to change, that is, the display state of the display object is the target display state. At this time, a mark may be added to the check-in information of the corresponding target object in the second display area, for example, a “VIP” mark may be added to the check-in information of the target object in the second display area.

Wherein, before the display state of the check-in information of the target object is changed, the photos included in the check-in information may be pictures stored in the database in advance. When marking the sign-in information, you can also intercept a photo of the target object from the video screen, and replace the photo of the target object in the sign-in information with the intercepted photo. The intercepted photos can be taken randomly, or taken from a frame of video image with the highest definition.

In a possible implementation manner, the position in the second display area is limited, and the number of target objects is multiple. In this case, the check-in information of the target object may be displayed scrolling, or, after a period of time after adding a mark to the check-in information of the target object, the check-in information of the target object may not be displayed in the second display area.

Based on the above solution, the automatic sign-in of the target object can be realized, the sign-in steps can be simplified, and the sign-in efficiency can be improved; by marking the sign-in information of the target object, the display method of the sign-in information can be enriched, and the target object whose display state is the target display state can be distinguished , And display status are not the target audience of the target display status.

Case 4: In the case of detecting that the target object is included in the video screen, the description information of the target object may be displayed in the third display area of the display screen.

The description information of the target object may be the attribute information of the target object and the face image of the target object intercepted from the video screen.

In different scenarios, the attribute information of the target object can be different. For example, in a classroom scenario, the attribute information of the target object can include student ID, gender, age, number of classes, etc.; while in public places such as exhibitions and shopping malls, the target object’s attribute information The attribute information may include identification, gender, number of visits, length of stay, etc.

The description information of the target object that can be displayed in the third display area is limited. For example, the third display area can only display the description information of N target objects. When the number of target objects included in the video screen is less than or equal to N, the third display area can display the description information of all target objects; When the number of target objects included in the screen is greater than N, for example, the video screen includes N+1 target objects, the description information of the target object displayed first can be deleted, and the latest N+1th target object can be displayed The description information of the target object that appears in the video screen, N is a positive integer.

In a possible implementation manner of this application, the third display area may also only display the description information of the first M target objects appearing in the video screen, and the description information of the target objects appearing after the Mth position in the video screen is not displayed. Show that M is a positive integer, and M and N can be equal or unequal. Or, display the description information of the first M target objects that meet the characteristics of a specific attribute.

In the case that the display object in the first display area is controlled to change the display state, and it is detected that the changed display state is the target display state, the description information of the target object displayed in the third display area may also be marked.

Exemplarily, after detecting that the target object is included in the video image, the flower special effect and the prompt "smile becomes a VIP" can be displayed in the corresponding position. After detecting the smile of the target object, you can change the color of the flower special effect (that is, the display state of the display object in the first display area becomes the target display state), and then add the description information of the target object displayed in the third display area. VIP" mark, you can also change the background color of the description information from the initial color to the target color.

In the above solution, the description information of the target object can be displayed, and after the display state of the display object is changed to the target display state, a mark is added to the description information, which enriches the display method of the description information and increases the interaction with the user.

Case 5: In the case where it is detected that the target object is included in the video screen, the business content can be displayed in the fourth display area of the display screen.

Among them, the business content displayed in the fourth display area may be, for example, advertisements, promotional videos, or venue introduction information, etc. The form of the business content is not limited to pictures, text, audio, video, etc.

Based on the cumulative detection result of the target object in the video screen, controlling the display state of the display object in the display screen may include: in the case of detecting that the posture of the target object in the video screen is switched from the first posture to the second posture, controlling the change The business content displayed in the fourth display area is switched from the first business content to the second business content.

Among them, the current posture of the target object is the first posture, and the preset posture is the second posture; the business content currently displayed in the fourth display area is the first business content, and the business content other than the first business content is the second Business content.

An indicator may be displayed in the fourth display area for instructing the target object to make a second gesture to switch the business content. For example, the indicator may be "please wave your hand to switch the display content". The current posture of the target object is the first posture. When it is detected that the target object makes a waving action, that is, when it is detected that the target object is switched from the first posture to the second posture, control the display in the fourth display area The business content is switched.

In the above solution, when a change in the posture of the target object is detected, the business content displayed in the fourth display area can be controlled to switch. In this way, the interaction with the target object is increased, and the switching method of business content is enriched.

Case 6: In a case where it is detected that the video frame includes at least one target object, the feature distribution information of the at least one target object is displayed in the fifth display area of the display screen.

Among them, the feature distribution information may include, but is not limited to: age distribution, gender distribution, attendance distribution, popular route distribution, regional population density distribution, etc.

Among them, when the feature distribution information includes the distribution of popular routes, the electronic device can be connected to multiple camera devices (the connection method can be wired connection or wireless connection); the movement route of each target object can be determined first, and then the moving route of each target object can be determined. Routes filter out popular routes and display them.

Wherein, as shown in FIG. 5, the method for determining the movement route of the target object may include steps S501 to S504.

S501. Acquire collected data of the first camera device, where the collected data includes the collected face image and identification information of the first camera device.

S502: Acquire location description information corresponding to the identification information of the first camera device.

S503: Determine the movement data of the target object identified by the face image based on the position description information of the first camera device.

S504. Use the movement data to determine the movement route of the target object.

Specifically, the first camera device may be a camera device that currently photographs the target object, and the collected data of the first camera device also includes the collection time of the face image; In the case of the historical movement data of the target object, the historical movement data may be updated based on the position description information of the first camera device to obtain the movement data; the setting before the collection time is detected If the historical movement data of the target user is not obtained within the time period, the position description information of the first camera device is used as the movement data. The historical movement data includes position description information of at least one second camera device, where the second camera device is a camera device that collects a face image of the target object within a set time period before the collection time.

In a possible implementation manner, the controlling the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video screen may include: based on the cumulative detection of at least one target object As a result, the control updates the feature distribution information displayed in the fifth display area.

Exemplarily, if the feature distribution information includes gender distribution, and the current video screen includes 5 male target objects and 2 female target objects, the displayed feature distribution information may show that the ratio of male to female is 5:2; or the gender distribution is displayed At the same time, the display can be performed in the form of a pie chart, and the display mode is not limited in the embodiment of the present disclosure.

In the above manner, by displaying and updating the feature distribution information in the fifth display area, statistics and analysis of the target object in the video picture can be realized.

Exemplarily, when it is detected that the target object is included in the video screen, if the display object associated with the target object is displayed and includes the above cases 1 to 6, the distribution of each display area in the display screen can be as shown in FIG. 6 Wherein, the first display area and the fourth display area may be the same area, that is, the acquired video picture, the first display object, and the display service content are displayed in different display modes.

In the case that the first display area and the fourth display area are different areas, that is, when video images and business content are displayed on the display screen at the same time, the distribution of each display area in the display screen can also be exemplified as shown in FIG. 7 .

Those skilled in the art can understand that in the above method of the specific implementation, the writing order of the steps does not mean a strict execution order but constitutes any limitation on the implementation process. The specific execution order of each step should be based on its function and possibility. The inner logic is determined.

Based on the same concept, the embodiment of the present disclosure also provides a display object control device. As shown in FIG. 8, the structure diagram of the display object control device provided by the embodiment of the present disclosure includes a first display module 801, The second display module 802 and the control module 803 are specifically:

The first display module 801 is used to display the acquired video picture on the display screen;

The second display module 802 is configured to display the display object associated with the target object on the display screen when it is detected that the target object is included in the video frame;

The control module 803 is configured to control the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame.

In a possible implementation manner, the second display module 802 may be used to display the initial display in the first display area of the display screen when the target object is detected in the video screen. The first display object in the state; wherein the initial display state is used to prompt the target object to make a target gesture.

In a possible implementation manner, the control module 803 may be used to: detect the target gesture made by the target object in the video frame; or based on the accumulated time or length of the target gesture made by the target object The cumulative number of times controls the change of the display state of the first display object in the first display area.

In a possible implementation manner, the control module 803 may be used to control the first display object to change the display state to an intermediate display state when the accumulated duration or the accumulated number of times has not reached a set value; Wherein, the intermediate display state changes with the cumulative value of the cumulative duration or the cumulative number of times to indicate the cumulative value.

In a possible implementation manner, the control module 803 may be configured to control the first display object to change the display state to the first target display state when the cumulative duration or cumulative number of times reaches a set value. , To indicate that the target posture made by the target object has met the set conditions.

In a possible implementation manner, the second display module 802 may be used to: detect the attribute characteristics of the target object in the video frame; display the attributes and characteristics of the target object in the first display area of the display screen. The corresponding second display object.

In a possible implementation manner, the control module 803 may be configured to: in the case of detecting that the target attribute characteristics of the target object change, control the second display object in the first display area to follow The change of the attribute characteristics changes the display state.

In a possible implementation manner, the second display object includes at least one of the following information: the identity of the target object, age value, smile value, charm value, viewing time, duration of different emotions, attention duration.

In a possible implementation manner, the second display module 802 may be used to display the target object in the second display area of the display screen in the case of detecting that the video frame includes the target object. The sign-in information of the target object.

In a possible implementation manner, the control module 803 may be further configured to: control the display object in the first display area of the display screen to change the display state, and detect that the changed display state is the target display In the case of the state, mark the sign-in information of the target object displayed in the second display area.

In a possible implementation manner, the second display module 802 may be used to display the target object in the third display area of the display screen in the case where it is detected that the video frame includes the target object. Descriptive information of the target object.

In a possible implementation manner, the control module 803 may be further configured to: control the display object in the first display area of the display screen to change the display state, and detect that the changed display state is the target display In the case of the state, marking the description information of the target object displayed in the third display area is performed.

In a possible implementation manner, the second display module 803 may be configured to display business content in the fourth display area of the display screen in the case of detecting that the target object is included in the video screen .

In a possible implementation manner, the control module 803 may be configured to: in the case of detecting that the posture of the target object in the video frame is switched from the first posture to the second posture, control the fourth posture to be The business content displayed in the display area is switched from the first business content to the second business content.

In a possible implementation manner, the second display module 802 may be configured to display in the fifth display area of the display screen when it is detected that the video frame includes at least one of the target objects At least one feature distribution information of the target object.

In a possible implementation manner, the control module 803 may be configured to: based on a cumulative detection result of at least one of the target objects, control to update the feature distribution information displayed in the fifth display area.

In some embodiments, the functions or templates contained in the device provided in the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. For specific implementation, refer to the description of the above method embodiments. For brevity, here No longer.

Based on the same technical concept, the embodiments of the present disclosure also provide an electronic device. Referring to FIG. 9, the electronic device may include: a processor 901, a memory 902, and a bus 903. Among them, the memory 902 includes a memory 9021 and an external memory 9022. The memory 9021 here is also called internal memory, which is used to temporarily store the calculation data in the processor 901 and the data exchanged with the external memory 9022 such as a hard disk. The processor 901 exchanges data with the external memory 9022 through the memory 9021. When the electronic device 900 is running, the processor 901 and the memory 902 communicate through the bus 903. The memory 902 stores machine-readable instructions executable by the processor 901. When the machine-readable instructions are executed by the processor 901, the processor 901 can implement any of the above-mentioned methods for displaying movement state information.

For the specific processing performed by the processor 901, reference may be made to the description in the foregoing method embodiment, and the description is not repeated here.

In addition, the embodiments of the present disclosure also provide a computer-readable storage medium with a computer program stored on the computer-readable storage medium, wherein, when the computer program is executed by a processor, the processor can implement any of the above Display the control method of the object.

The computer program product of the method for controlling the display object provided by the embodiment of the present disclosure includes a computer-readable storage medium storing program code. The program code includes instructions that can be used to execute the display object described in the above method embodiment. For the steps of the control method, refer to the foregoing method embodiment for details, which will not be repeated here.

Those skilled in the art can clearly understand that, for the convenience and conciseness of the description, the specific working process of the system and device described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device, and method may be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For another example, multiple units or components may be combined or integrated into another system, or some features may be omitted or not implemented. In addition, the displayed or discussed mutual coupling or direct coupling or communication connection may be indirect coupling or communication connection through some communication interfaces, devices or units, and may be in electrical, mechanical or other forms.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the embodiments of the present disclosure.

In addition, the functional units in the various embodiments of the present disclosure may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit.

If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the embodiments of the present disclosure can be embodied in the form of software products in essence, or a part that contributes to the prior art, or a part of the embodiment. The computer software product is stored in a storage medium and includes a number of instructions to enable a computer device (which may be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present disclosure. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disks or optical disks and other media that can store program codes. .

The above are only some embodiments of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art based on the present disclosure should be included in the scope of the present disclosure. Therefore, the scope of the present disclosure is determined by the claims.

Claims

A control method for display objects, including:

Display the acquired video picture on the display screen;

In a case where it is detected that the video screen includes a target object, display the display object associated with the target object on the display screen;

Based on the cumulative detection result of the target object in the video frame, controlling the display object in the display screen to change the display state.
The method according to claim 1, wherein said displaying the display object associated with the target object on the display screen in the case where it is detected that the target object is included in the video picture comprises:

In a case where it is detected that the target object is included in the video screen, displaying the first display object in the initial display state in the first display area of the display screen,

Wherein, the initial display state is used to prompt the target object to make a target gesture.
3. The method according to claim 2, wherein the controlling the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame comprises:

Detecting the target posture made by the target object in the video frame;

Controlling the first display object in the first display area to change its display state based on the cumulative duration or cumulative number of times the target object makes the target gesture.
3. The method according to claim 3, wherein said controlling said first display object in said first display area to change its display state based on the cumulative duration or cumulative number of times the target posture is made by said target object, include:

In the case that the cumulative duration or cumulative number of times has not reached the set value, controlling the first display object to change the display state to the intermediate display state,

Wherein, the intermediate display state changes with the cumulative value of the cumulative duration or the cumulative number of times to indicate the cumulative value.
3. The method according to claim 3, wherein said controlling said first display object in said first display area to change its display state based on the cumulative duration or cumulative number of times the target posture is made by said target object, include:

When the cumulative duration or cumulative number of times reaches a set value, the first display object is controlled to change the display state to the first target display state, so as to indicate that the target posture made by the target object has met the set value. Set conditions.
The method according to claim 1, wherein said displaying the display object associated with the target object on the display screen in the case where it is detected that the target object is included in the video picture comprises:

In a case where it is detected that the target object is included in the video picture, detecting the attribute characteristics of the target object in the video picture;

The second display object corresponding to the attribute feature is displayed in the first display area of the display screen.
The method according to claim 6, wherein the controlling the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame comprises:

In the case of detecting that the attribute characteristic of the target object changes, controlling the second display object in the first display area to change the display state according to the change of the attribute characteristic.
The method according to claim 6 or 7, wherein the second display object includes at least one of the following information:

The identity, age value, smile value, charm value, viewing duration, duration of different emotions, and attention duration of the target object.
8. The method according to any one of claims 1 to 8, wherein the display object associated with the target object is displayed on the display screen in the case of detecting that the target object is included in the video screen ,include:

In a case where it is detected that the target object is included in the video screen, the check-in information of the target object is displayed in the second display area of the display screen.
The method according to claim 9, further comprising:

In the case that the display object in the first display area of the display screen is controlled to change its display state, and it is detected that the changed display state is the target display state, mark the target displayed in the second display area The sign-in information of the object.
The method according to any one of claims 1 to 10, wherein said displaying the display object associated with the target object on the display screen in the case where it is detected that the video frame includes a target object, comprises :

In the case where it is detected that the target object is included in the video frame, the description information of the target object is displayed in the third display area of the display screen.
The method according to claim 11, the method further comprising:

In the case that the display object in the first display area of the display screen is controlled to change its display state, and it is detected that the changed display state is the target display state, mark the target displayed in the third display area Descriptive information of the object.
The method according to any one of claims 1 to 12, wherein the display object associated with the target object is displayed on the display screen when the target object is detected in the video screen ,include:

In a case where it is detected that the target object is included in the video picture, the business content is displayed in the fourth display area of the display screen.
The method according to claim 13, wherein the controlling the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame comprises:

In the case of detecting that the posture of the target object in the video screen is switched from the first posture to the second posture, control to switch the business content displayed in the fourth display area from the first business content to the second posture 2. Business content.
The method according to any one of claims 1 to 14, wherein the display object associated with the target object is displayed on the display screen in the case of detecting that the target object is included in the video screen ,include:

In a case where it is detected that the video frame includes at least one target object, the feature distribution information of at least one target object is displayed in a fifth display area of the display screen.
The method according to claim 15, wherein the controlling the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame comprises:

Based on the cumulative detection result of at least one of the target objects, control to update the feature distribution information displayed in the fifth display area.
A display object control device, including:

The first display module is used to display the acquired video picture on the display screen;

The second display module is configured to display the display object associated with the target object on the display screen when it is detected that the target object is included in the video frame;

The control module is configured to control the display object in the display screen to change the display state based on the cumulative detection result of the target object in the video frame.
The device according to claim 17, wherein the second display module is configured to: in the case of detecting that the target object is included in the video frame, display the initial display in the first display area of the display screen. The first display object in the display state,

Wherein, the initial display state is used to prompt the target object to make a target gesture.
The device according to claim 18, wherein the control module is used to:

Detecting the target posture made by the target object in the video frame;

Controlling the first display object in the first display area to change its display state based on the cumulative duration or cumulative number of times the target object makes the target gesture.
The device according to claim 19, wherein the control module is configured to:

In the case that the cumulative duration or cumulative number of times has not reached the set value, controlling the first display object to change the display state to the intermediate display state,

Wherein, the intermediate display state changes with the cumulative value of the cumulative duration or the cumulative number of times to indicate the cumulative value.
The device according to claim 19, wherein the control module is configured to:

When the cumulative duration or cumulative number of times reaches a set value, the first display object is controlled to change the display state to the first target display state, so as to indicate that the target posture made by the target object has met the set value. Set conditions.
The device according to claim 17, wherein the second display module is used for:

In a case where it is detected that the target object is included in the video picture, detecting the attribute characteristics of the target object in the video picture;

The second display object corresponding to the attribute feature is displayed in the first display area of the display screen.
The device according to claim 22, wherein the control module is configured to:

When it is detected that the target attribute characteristic of the target object changes, the second display object in the first display area is controlled to change the display state according to the change of the attribute characteristic.
The device according to claim 22 or 23, wherein the second display object includes at least one of the following information:

The identity, age value, smile value, charm value, viewing duration, duration of different emotions, and attention duration of the target object.
The device according to any one of claims 17 to 24, wherein the second display module is used for:

In a case where it is detected that the target object is included in the video screen, the check-in information of the target object is displayed in the second display area of the display screen.
The device according to claim 25, wherein the control module is further used for:

In the case that the display object in the first display area of the display screen is controlled to change its display state, and it is detected that the changed display state is the target display state, mark the target displayed in the second display area The sign-in information of the object.
The device according to any one of claims 17 to 26, wherein the second display module is used for:

In the case where it is detected that the target object is included in the video frame, the description information of the target object is displayed in the third display area of the display screen.
The device according to claim 27, wherein the control module is further used for:

In the case that the display object in the first display area of the display screen is controlled to change its display state, and it is detected that the changed display state is the target display state, mark the target displayed in the third display area Descriptive information of the object.
The device according to any one of claims 17 to 28, wherein the second display module is used for:

In a case where it is detected that the target object is included in the video screen, the business content is displayed in the fourth display area of the display screen.
The device according to claim 29, the control module is configured to:

In the case of detecting that the posture of the target object in the video screen is switched from the first posture to the second posture, control to switch the business content displayed in the fourth display area from the first business content to the second posture 2. Business content.
The device according to any one of claims 17 to 30, wherein the second display module is used for:

In a case where it is detected that the video frame includes at least one target object, the feature distribution information of at least one target object is displayed in a fifth display area of the display screen.
The device according to claim 31, wherein the control module is configured to:

Based on the cumulative detection result of at least one of the target objects, control to update the feature distribution information displayed in the fifth display area.
An electronic device including:

Processor; and

Memory,

Wherein, the memory stores machine-readable instructions executable by the processor, and when the machine-readable instructions are executed by the processor, the processor implements any one of claims 1 to 16 Display the control method of the object.
A computer-readable storage medium having a computer program stored on the computer-readable storage medium, wherein when the computer program is executed by a processor, the processor realizes the display object according to any one of claims 1 to 16的控制方法。 Control methods.