JP2022520512A - Data push methods, devices, electronic devices, computer storage media, and computer programs - Google Patents

Abstract

The embodiments of the present invention propose a data push method, an apparatus, an electronic device, a computer storage medium, and a computer program, wherein the method includes a step of acquiring a multi-frame person image collected by an image acquisition apparatus, and the above-mentioned step. A step of determining the staying time in front of the face information display device in the multi-frame person image based on the multi-frame person image, a step of determining a candidate input target based on the staying time, and the candidate. It includes a step of determining recommendation information based on an input target and displaying the recommendation information via the information display device. [Selection diagram] Fig. 1

Description

The present invention relates to computer vision technology, in particular to data push methods, devices, electronic devices, computer storage media, and computer programs.

<Mutual citation of related applications>
The present invention has a filing date of January 17, 2020, an application number of 20100054747.0, and a Chinese patent application whose title is "data push method, apparatus, electronic device, and computer storage medium". Claiming priority, the entire content of the Chinese patent application is incorporated herein by reference.

In the background of the new retail industry, the introduction of recommendation information such as advertisements into offline business situations is becoming more and more popular. Taking the introduction of advertisements to major shopping malls as an example, many recommendation information currently includes promotional content of shopping malls and advertising content of partners, but due to the diversification of the audience, the recommendation information is Often, the ideal input effect cannot be obtained, resulting in wasted input resources.

The embodiments of the present invention provide a data push method, wherein the method is:
A step to acquire a multi-frame portrait image collected by an image collector, and
A step of determining the time spent in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image, and
The step of determining the candidate input target based on the staying time and
It includes a step of determining recommendation information based on the candidate input target and displaying the recommendation information via the information display device.

Optionally, the step of determining a candidate input target based on the staying time is a person corresponding to a face having a staying time of a predetermined time or longer, and / or a face having a staying time within a predetermined range. Includes a step to target the person corresponding to.

Optionally, the imaging range of the image acquisition device includes the front area of the information display device.
The image collecting device is deployed on the information display device, or the distance between the image collecting device and the information display device is equal to or less than a predetermined distance threshold value.

The step of selectively determining the recommendation information based on the candidate input target is
It is a step of determining at least one candidate input target among the candidate input targets as the final input target based on the distance between each candidate input target among the candidate input targets and the information display device. The step that the distance between the final input target and the information display device is equal to or less than a predetermined distance,

Includes a step of determining the recommendation information for the final input target.

The step of selectively determining the recommendation information based on the candidate input target is
This is a step of determining at least one candidate input target among the candidate input targets as the final input target based on the face size of each candidate input target in the target person image. The pixel size corresponding to the face in the target person image of the input target is equal to or larger than a predetermined size, or the reference area in the target person image of the area corresponding to the face in the target person image of the final input target. Steps where the ratio to is greater than or equal to the predetermined ratio,
Includes a step of determining the recommendation information for the final input target.

Optionally, the method is:
A step of determining the predetermined size based on the image acquisition resolution of the image acquisition device, wherein the predetermined size further includes a step having a positive correlation with the image acquisition resolution.

Optionally, based on the multi-frame person image, the step of determining the time spent in front of the face information display device in the multi-frame person image is
A step of determining the time spent in front of the information display device of a face in the plurality of frames of a person image by performing face tracking on the plurality of frames of the person image is included.

Optionally, the method is:
The step of determining the face quality in each person image among the multiple frames of the person image, and
Further including a step of selecting one frame image having the highest face quality from the images corresponding to the candidate input target.

The step of selectively determining the recommendation information based on the candidate input target includes a step of determining the recommendation information based on the face image of the candidate input target corresponding to the one-frame image having the highest face quality.

The embodiments of the present invention further provide a data push device, wherein the device.
An acquisition module for acquiring multiple frames of human images collected by an image collection device,
A determination module for determining the time spent in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image, and a determination module.
A processing module for determining a candidate input target based on the staying time, determining recommendation information based on the candidate input target, and displaying the recommendation information via the information display device is provided.

When selectively determining a candidate input target based on the dwell time, the processing module specifically
It is configured to target a person corresponding to a face having a staying time of a predetermined time or longer and / or a person corresponding to a face having a staying time within a predetermined range as a candidate input target.

Optionally, the imaging range of the image acquisition device includes the front area of the information display device.
The image collecting device is deployed on the information display device, or the distance between the image collecting device and the information display device is equal to or less than a predetermined distance threshold value.

When selectively determining recommendation information based on the candidate input target, the processing module specifically
Based on the distance between each candidate input target among the candidate input targets and the information display device, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is determined. The distance between the information display device and the information display device is equal to or less than a predetermined distance.
It is configured to determine the recommendation information for the final input target.

When selectively determining recommendation information based on the candidate input target, the processing module specifically
Based on the face size of each candidate input target among the candidate input targets in the target person image, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is said to be the same. The pixel size corresponding to the face in the target person image is equal to or larger than a predetermined size, or the ratio of the area corresponding to the face in the target person image of the final input target to the reference area in the target person image is determined. It is above the specified ratio and
It is configured to determine the recommendation information for the final input target.

Optionally, the processing module further determines the predetermined size based on the image acquisition resolution of the image acquisition device, which has a positive correlation with the image acquisition resolution of the image acquisition device. It is configured as follows.

When selectively determining the time spent in front of the face information display device in the multi-frame person image based on the multi-frame person image, the determination module specifically comprises.
By performing face tracking on the plurality of frames of the person image, it is configured to determine the time of stay of the face in the plurality of frames of the person image in front of the information display device.

Optionally, the decision module further
The face quality in each person image among the multiple frames of person images is determined.
It is configured to select one frame image having the highest face quality from the images corresponding to the candidate input target.

Optionally, the processing module is specifically configured to determine recommendation information based on the face image of the candidate input target corresponding to the one-frame image with the highest face quality.

An embodiment of the present invention further provides an electronic device, wherein the electronic device comprises a processor and a memory for storing a computer program that can be executed by the processor.
When the processor executes the computer program, the data push method according to any one of the above is executed.

An embodiment of the present invention further provides a computer storage medium in which a computer program is stored, and when the computer program is executed by the processor, the data push method according to any one of the above can be used by the processor. It will be realized.

The embodiments of the present invention further provide a computer program, and when the computer program is executed by the processor, the processor realizes the data push method according to any one of the above.

In the embodiment of the present invention, the staying time in front of the information display device is an important reference factor for determining whether or not the user is paying attention to the recommended information, and therefore the staying time in front of the face information display device is used. The method of determining the candidate input target based on time allows the input target of the recommendation information to be selected with the aim, the effectiveness of the recommendation information push is further increased, the waste of input resources is reduced, and further, the input resource is wasted. It is advantageous for improving the input / output ratio of recommendation information.

It should be understood that the general description above and the detailed description below are merely exemplary and interpretive and are not limitations to the present invention.

The drawings herein are incorporated into the specification and form part of the specification. These drawings show examples conforming to the present invention and are used together with the specification to explain the examples of the present invention.
It is a flowchart of the data push method of the Example of this invention. It is a schematic diagram of one application scene of the Example of this invention. It is a schematic diagram of another application scene of the Example of this invention. It is a schematic diagram of the structure of the data push device of the Example of this invention. It is a schematic diagram of the structure of the electronic device of the Example of this invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings and examples. It should be understood that the examples provided herein are merely for the purpose of interpreting the present invention and do not limit the present invention. Further, the examples provided below are not all the examples for carrying out the present invention, but are some examples for carrying out the present invention, and if there is no contradiction, the examples of the present invention are used. Can be carried out in any combination.

In an embodiment of the invention, a method comprising a set of elements, such as "contains" and "comprises", or any other variation thereof, intended to cover the non-exclusive "contains". Alternatively, it should be explained that the device includes not only the clearly described elements, but also other elements that are not clearly described, or elements that are specific to the practice of the method or device. Without further limitation, elements limited by a phrase such as "contains one ..." do not exclude the method containing the element or other related elements further present in the device (eg, method). A step in or a unit in a device, for example, a unit in a device can be some circuits, some processors, some programs or software, etc.).

As used in the present invention, terms such as "and / or" are used solely to describe the relationships of related objects and mean that three types of relationships can exist, eg, A and. / Or B, which indicates that there are three cases, such as when A is present alone, when A and B are present at the same time, and when B is present alone. Also, terms such as "at least one" as used in the present invention mean any one of the elements or any combination of at least two of the elements. For example, at least one of A, B, C means any one or more elements selected in a set composed of A, B, and C.

For example, the data push method provided by the embodiments of the present invention includes a series of steps, but the data push method provided by the embodiments of the present invention is not limited to the described steps, and similarly, the present invention. The data push device provided by the embodiments of the present invention includes a series of modules, but the data push device provided by the embodiments of the present invention is not limited to the inclusion of the clearly described modules, and is further related. It may include modules that need to be deployed when retrieving information or performing processing based on the information.

The embodiments of the present invention can be applied to hardware such as terminals, servers, or computer systems composed of hardware to operate with many other general purpose or dedicated computing system environments or configurations. Alternatively, an embodiment of the present invention can be realized by a method in which a processor executes computer executable code. Here, the terminal can be a thin client, a thick client, a handheld or laptop device, a system based on a microprocessor, a settop box, a programmable consumer electronics product, a network personal computer, a small computer system, etc., and the server is a server. It can be a decentralized cloud computing environment including computer systems, small computer systems, large computer systems, and any of the above systems.

Electronic devices such as terminals and servers can be described in the general context of computer system executable instructions (eg, program modules) executed by the computer system. In general, a program module may include routines, programs, target programs, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. The computer system / server may be implemented in a decentralized cloud computing environment, in which tasks are performed by remote processing devices linked over a communication network. In a distributed cloud computing environment, program modules can be located on the storage media of local or remote computing systems, including storage devices.

In the offline business scene, more and more hardware devices are beginning to have the ability to collect images, and these hardware devices can effectively reach people in areas such as shopping malls, stores, etc., to the characteristics of different people. Offers the possibility of "new retail" to push different recommendation information based on. However, the scene of the flow of people collected by the camera is complicated, and multiple people often appear at the same time in the recorded video, and such a complicated environment makes it difficult to select the target for inputting recommendation information. Increased the degree.

In some embodiments of the present invention, a data push method is proposed, and the embodiments of the present invention may be applied to situations where it is necessary to execute recommendation information input such as advertisement input and offline retailing.

FIG. 1 is a flowchart of a data push method according to an embodiment of the present invention, and as shown in FIG. 1, the data push method may include steps 101 to 104.

In step 101, a plurality of frames of human images collected by the image collecting device are acquired.

In an embodiment of the invention, the image acquisition device refers to a device for collecting a video stream or an image, for example, the image collection device may be a video detection device or collects an image such as a capture camera. It may be a device.

The image collecting device may be arranged around the information display device, the shooting range of the image collecting device may include the front area of the information display device, and the front area of the information display device is the display content of the information display device. It is an area where you can see. Illustratively, the front area of the information display device can be the area directly in front of the information display device and / or the side front area, and the embodiments of the present invention are not limited thereto.

In the embodiment of the present invention, the information display device is a device that displays recommended information. Illustratively, the information display device can be a display screen or other type of display device. The format in which the information display device displays the recommendation information may include, but is not limited to, a method of displaying by an image, text, a list, or the like. In the actual display process, the recommendation information can be displayed in a static or dynamic display method. Here, the method of dynamically displaying includes, but is not limited to, switching of recommendation information periodically or irregularly, and includes, for example, switching of recommendation information in a format such as scrolling, page turning, and highlighting. , Not limited to these.

In one embodiment, the image collecting device is deployed on the information display device, and in this way, the shooting range of the image collecting device includes the area of the position where the display content of the information display device is viewed, and the above information. Combined with the description of the front area of the display device, the embodiments of the present invention are advantageous for collecting a person image in the front area of the information display device of the image collection device.

In another embodiment, the distance between the image acquisition device and the information display device is equal to or less than a predetermined distance threshold. When the distance between the image collecting device and the information display device is smaller, it can be understood that the shooting range of the image collecting device is likely to include the area of the position where the display content of the information display device is viewed. Combined with the above description of the front area of the information display device, the embodiments of the present invention are advantageous for collecting a person image in the front area of the information display device of the image collection device.

Here, the predetermined distance threshold has a positive correlation with the shooting range of the image collecting device, and when the shooting range of the image collecting device is larger, the predetermined distance threshold is larger. Even when the distance between the image and the information display device is larger, the shooting range of the image acquisition device can include the front area of the information display device. When the shooting range of the image collecting device is smaller, the distance between the image collecting device and the information display device is smaller, which is advantageous for shooting the front area of the information display device of the image collecting device.

Here, when the image collecting device collects the video stream, a plurality of frames of a person image may be selected from all the image frames of the video stream by face recognition and / or human body identification. First, the video stream may be selected. After selecting a part of the image frames from all the image frames of the above, a plurality of frames of a person image may be selected from the above-mentioned part of the image frames by face recognition and / or human body identification. In some other embodiments, if the image acquisition device is a device for image acquisition such as a capture camera, multiple frames of portrait images may be obtained by multiple captures.

In step 102, the staying time in front of the face information display device in the plurality of frames of the person image is determined based on the plurality of frames of the person image.

In actual application, face detection may be performed on a person image in each frame, whereby face tracking on a person image in a plurality of frames can be realized. In the embodiment of the present invention, the embodiment of face detection and face tracking is not limited.

In one embodiment, by performing face tracking on a plurality of frames of a person image, it is possible to determine the staying time of the face in the plurality of frames of the person image in front of the information display device. Specifically, by performing face tracking on a multi-frame portrait, multiple frames are collected based on the collection time interval after determining the same face in the different frame portrait images. It is possible to determine the length of stay in front of a facial image collector for a person image of a frame.

In step 103, a candidate input target is determined based on the staying time.

Specifically, it is possible to select a candidate input target from the persons corresponding to the faces in the person image of a plurality of frames, based on the staying time in front of the face information display device in the person image of a plurality of frames.

In the case of the realization form of this step, in one example, from among the persons corresponding to the faces in the person image of a plurality of frames, the person corresponding to the face whose stay time is longer than a predetermined time may be set as the candidate input target. can.

In the embodiments of the present invention, the predetermined time can be a default value or can be preset according to actual application requirements. Illustratively, if the information display device is located in a scene where the flow of people is relatively fast, a lower predetermined time can be set because the time for the person to pay attention to the information is limited. If the staying time is lower than a predetermined time, it can be considered that the corresponding person is effectively paying attention to the information display device. If the information display device is located in a scene where the flow of people is relatively slow, there is enough time for the person to pay attention to the information, so a higher predetermined time can be set, and in this way, the staying time can be set. If it is higher than a predetermined time, it can be considered that the corresponding person is effectively paying attention to the information display device. In a specific example, when the information display device is located in a scene where the flow speed of people is relatively high, such as a subway station, a predetermined time may be set to 3 seconds. When the information display device is located in a scene where the flow speed of people is relatively slow, such as in a bus or a building, a predetermined time may be set to 5 seconds.

Users who spend less time in front of the information display device are less likely to pay attention to the recommendation information, and users who spend more time in front of the information display device are more likely to pay attention to the recommendation information. Because it is expensive, if the staying time in front of the face information display device is longer than a predetermined time, targeting the person corresponding to the face as a candidate input target for recommendation information is aimed at inputting recommendation information. Understand that you can choose.

In the case of the realization form of this step, as another example, a person corresponding to a face whose stay time is within a predetermined range may be selected as a candidate input target from the persons corresponding to the faces in the person image of a plurality of frames. ..

In the embodiments of the present invention, the predetermined range may be the default range or may be preset according to the actual application requirements.

The upper limit, the lower limit, and the interval span of a predetermined range can all be set flexibly. Illustratively, the method of setting the lower limit value of the range can be the same as the method of setting the value of the above time, that is, the information display device is located in a scene where the speed of the flow of people is relatively high. If so, the lower limit of the predetermined range is lower, and if the information display device is located in a scene where the speed of the flow of people is relatively slow, the lower limit of the predetermined range is higher.

The upper limit of the predetermined range can be determined based on the speed of the flow of people in the actual scene. Illustratively, when the information display device is located in a scene where the flow speed of a person is relatively high, the time for a person to pay attention to the information is limited, so that a lower upper limit of a predetermined range can be set, and the staying time can be set. If it is equal to or higher than the lower upper limit of the predetermined range, it can be considered that the person merely stays in front of the information display device for a long time and does not pay attention to the display content of the information display device. When the information display device is located in a scene where the flow speed of a person is relatively slow, the person has enough time to pay attention to the information, so that a higher upper limit of a predetermined range can be set. If it is equal to or higher than the upper limit of the higher predetermined range, it can be considered that the person merely stays in front of the information display device for a long time and does not pay attention to the display content of the information display device. In a specific example, when the information display device is located in a scene where the flow speed of people is relatively high, such as a subway station, the upper limit of a predetermined range is set to 1 minute, and the information display device is in a bus or in a building. When the speed of the flow of people is relatively slow, such as inside, the upper limit of a predetermined range can be set to 2 minutes.

As an example of the method for determining the interval span in a predetermined range, the upper limit and the lower limit of the predetermined range can be determined first, and then the interval span in the predetermined range can be determined based on the above-mentioned contents. For example, if the lower limit of the predetermined range is 4 seconds and the upper limit of the predetermined range is 80 seconds, the interval span of the predetermined range is 76 seconds and the lower limit of the predetermined range is 6 seconds. Moreover, when the upper limit of the predetermined range is 120 seconds, the interval span of the predetermined range is 114 seconds.

Based on the above, a user whose stay time in front of the information display device is within a predetermined range is more likely to pay attention to the recommendation information, and the stay time in front of the information display device is within a predetermined range. Since users who do not belong to the recommendation information are less likely to pay attention to the recommendation information, if the staying time in front of the face information display device belongs to a predetermined range, the person corresponding to the face is selected as a candidate input target for the recommendation information. By doing so, it can be seen that the target for inputting the recommendation information can be selected with the aim.

In the actual application, the setting items for inputting the recommendation information can be set in advance before inputting the recommendation information, and the setting items are the predetermined time, the upper limit of the predetermined range, the lower limit of the predetermined range, and the setting items. , May include an interval span in a predetermined range.

In step 104, the recommendation information is determined based on the candidate input target, and the recommendation information is displayed via the information display device.

In order to aim and provide recommendation information to the relevant input target, the recommendation information may be determined in combination with the relevant data of the candidate input target and / or the final input target mentioned below. In one embodiment, the relevant data may include, but are not limited to, the face attributes of the input target. Here, the face attribute is used to represent the identification feature of the input target, and the identification feature may specifically include a combination of one or more terms given in the following example, and the example may include. For example, an identifier that can effectively classify different input targets such as face recognizers, at least one item in the basic information of the input target such as gender, age, and facial expression, and the total consumption amount and / or the number of times of consumption. It may be included. Of course, the status feature may further include, but is not limited to, content not mentioned in the example. In the embodiment of the present invention, the conditions and methods for selecting the recommended information are not limited.

In actual application, after determining the recommendation information, a display command can be sent to the information display device to instruct the information display device to display the recommendation information, and the information display device issues the display command. Later, the above recommendation information can be displayed.

In the actual application, steps 101 to 104 can be realized by utilizing the processor in the electronic device. The above-mentioned processors include application-specific integrated circuits (ASICs), digital signal processors (Digital Signal Processors, DSPs), digital signal processing devices (Digital Signal Processors, DSPs), digital signal processing devices (Digital Processing PLCs), and digital signal processing devices (Digital Process Logic Devices). It can be at least one of a PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microprocessor, and a microprocessor.

In the embodiment of the present invention, the staying time in front of the information display device is an important reference factor for determining whether or not the user is paying attention to the recommended information, and therefore the staying time in front of the face information display device is used. The method of determining the candidate input target based on time allows the input target of the recommendation information to be selected with the aim, the effectiveness of the recommendation information push is further increased, the waste of input resources is reduced, and further, the input resource is wasted. It can be seen that it is advantageous for improving the input / output ratio of recommended information.

Further, in the embodiment of the present invention, when a plurality of people are photographed by the image collecting device, the staying time in front of the image collecting device of each face can be effectively identified, that is, the embodiment of the present invention. The example can be adapted to input recommendation information for multiple scenes.

Regarding the implementation form in which the recommendation information is determined based on the candidate input target, in the first example, the candidate input target in the candidate input target is based on the distance between each candidate input target and the information display device. At least one candidate input target can be determined as the final input target, and the distance between the final input target and the information display device is less than or equal to a predetermined distance.

After obtaining the final input target, the recommendation information can be determined for the final input target.

In one embodiment, a predetermined distance can be predetermined, and the predetermined distance represents the maximum distance at which the user forms an effective attention to the display content of the information display device. For example, the predetermined distance is 0.5 m, 1 m, 2 m, 3 m, 5 m, or the like. In one possible implementation, the predetermined distance can be the distance between another candidate input target (ie, a candidate input target other than the final input target) and the information display device. In practical applications, the value of a given distance can be adjusted automatically, for example, selection of final input targets, such as the number of candidate input targets and the imaging resolution of the image acquirer for collecting candidate input targets. It can be determined based on, but is not limited to, a combination of one or more terms among many factors that may affect the.

After that, at least one candidate input target whose distance from the information display device is equal to or less than a predetermined distance is selected from each candidate input target, and at least one selected candidate input target is set as the final input target, and finally. Other input targets can be determined based on the input target. It can be seen that the distance between the other input target and the information display device is larger than the predetermined distance.

In actual application, the predetermined distance can be determined based on the display size of the information displayed by the information display device, and the predetermined distance has a positive correlation with the display size. Here, the information displayed by the information display device may be information such as characters and images. When the information displayed by the information display device is a character, the display size of the information displayed by the information display device can represent the size of the font of the character, and when the information displayed by the information display device is an image. The display size of the information displayed by the information display device can represent the size of the image. For example, if the font of the characters displayed by the information display device is an oversized font, the predetermined distance is 5 m, and if the font of the characters displayed by the information display device is a small font, the predetermined distance is 1 m. ..

If the display size of the information displayed by the information display device is larger, the user can effectively see the information at a longer distance, and if the display size of the information displayed by the information display device is smaller, the user is closer. Since the information can be effectively viewed by distance, it can be understood that the predetermined distance has a positive correlation with the above display size and that the predetermined distance matches the actual viewing needs.

If the distance between the candidate input target and the information display device is relatively small, it is more likely that the candidate input target is a person at an effective attention distance in front of the information display device, and the information display device It is even more likely that you have stayed before and are paying attention to the recommendation information. Therefore, the method of determining the final input target based on the distance between the candidate input target and the information display device can select the input target of the recommendation information with the aim, and the effectiveness of the recommendation information push. It can be seen that the cost becomes even higher and the waste of input resources can be reduced.

Regarding the implementation form in which the recommendation information is determined based on the candidate input target, in the second example, the candidate input target among the candidate input targets is based on the face size in the target person image of each candidate input target. At least one candidate input target can be determined as the final input target, and the pixel size corresponding to the face in the target person image of the final input target is equal to or larger than a predetermined size, or the target person image of the final input target. The ratio of the region corresponding to the face in the above to the reference region in the target person image is a predetermined ratio or more.

After obtaining the final input target, the recommendation information can be determined for the final input target.

In the embodiment of the present invention, the target person image represents an image including the candidate input target collected by the image collecting device, and the target person image may include one candidate input target or a plurality of candidates. An input target may be included, and the embodiments of the present invention are not limited thereto. Pixel size represents the size of the pixel value that the face occupies in the image, and exemplary, the pixel size may include the length and / or width of the pixels that the face occupies, depending on which the given size is. , Pixel length and / or width values may be included. When the length of the pixels occupied by the face is greater than or equal to the length value in the predetermined size and / or the width of the pixels occupied by the face is greater than or equal to the width value in the predetermined size, the pixel size corresponding to the face is determined. It can be considered to be greater than or equal to a predetermined size, otherwise the pixel size corresponding to the face can be considered to be less than a predetermined size.

In a specific embodiment, a face recognition algorithm is used to determine the extrinsic rectangle of the face along the edges of the face, and the face has pixel values for the length and / or width of the facial extrinsic rectangle. It can be the length and / or width of the pixels it occupies.

In one embodiment, a predetermined size is determined based on a predetermined distance and / or an image acquisition resolution of an image acquisition device. A given size has a negative association with a given distance and / or a given size has a positive correlation with the image acquisition resolution of the image acquisition device.

Here, the image collection resolution of the image collection device represents the resolution of the collected image, and the higher the image collection resolution of the image collection device, the pixel length value and / or the pixel width of the image collected by the image collection device. The value is large.

If the predetermined distance is larger, the distance between the candidate input target and the information display device may be larger, and at this time, due to the image collection characteristics of the image collection device, the pixel size corresponding to the face in the target person image Since is smaller, setting a smaller predetermined size is more suitable for the actual needs, and if the predetermined distance is smaller, the distance between the candidate input target and the information display device may be smaller. At this time, due to the image collection characteristics of the image collection device, the pixel size corresponding to the face in the target person image is larger, so setting a larger predetermined size is more suitable for the actual needs. Can understand.

When the image collection resolution of the image collector is smaller, it is more practical to set a smaller predetermined size because the image acquisition characteristics of the image collector have a smaller pixel size corresponding to the face in the target person image. If the image collection device meets your needs and the image collection resolution of the image collection device is larger, set a larger predetermined size because it is larger than the pixel size corresponding to the face in the target person image due to the image collection characteristics of the image collection device. Is more suited to the actual needs.

In one example, the reference area represents the entire area of the target person image, and the area of the target person image may be determined by the shooting range of the image collecting device. For example, the entire area of the target person image may be a single image including the composition of the human body, the ground, buildings, animals, plants, etc., and the entire area of the target person image can be represented by the pixel size of the image. For example, the entire area of the target person image is an area of 3800 * 2450 (length * width), that is, the pixel length value of the entire target person image is 3800 and the pixel width value of the entire target person image is 2450. Is.

In another example, the reference area represents the sum of all face areas in the entire target person image, and the entire target person image may include images of one or more faces. In an actual embodiment, after collecting the entire target person image, the face recognition algorithm can be used to detect the size of each face area in the entire target person image, and the total is all in the entire target person image. The total of the facial areas of can be calculated.

In one embodiment, a predetermined ratio is determined based on a predetermined distance, and the predetermined ratio has a negative association with the predetermined distance. For example, when the predetermined distance is 5 m, the predetermined ratio is 10%, and when the predetermined distance is 1 m, the predetermined ratio is 30%.

If the predetermined distance is larger, the distance between the candidate input target and the information display device may be larger, and at this time, due to the image collection characteristics of the image acquisition device, the face in the target person image of the final input target Since the ratio of the corresponding area to the above reference area may be smaller, setting a smaller predetermined ratio is more suitable for the actual needs, and if the predetermined distance is smaller, the candidate is input. The distance between the target and the information display device may be smaller, and at this time, due to the image collection characteristics of the image acquisition device, the area corresponding to the face in the target person image of the final input target with respect to the above reference area. It can be understood that setting a larger predetermined ratio is more suited to the actual needs, as the ratio can be larger.

When the pixel size corresponding to the face in the target person image of the candidate input target is relatively large, or when the ratio of the area corresponding to the face in the target person image of the candidate input target to the above reference area is relatively large. , The candidate input target is more likely to be a person at an effective attention distance in front of the information display device, and more likely to stay in front of the information display device and pay attention to the recommendation information. Therefore, in the method of determining the final input target based on the face size in the target person image of the target input target of the above candidate input target, the input target of the recommendation information can be selected with the aim, and the effectiveness of the recommendation information push is effective. It will be even higher and you will find that you can reduce the waste of input resources.

Regarding the implementation form in which the recommendation information is determined based on the candidate input target, in the third example, at least one candidate input target among the candidate input targets is determined as the final input target based on the first factor and the second factor. can do. Here, the first factor represents the distance between each candidate input target among the above candidate input targets and the information display device, and the second factor is the distance between each candidate input target among the above candidate input targets. , Represents the face size in the target person image.

In one example, a candidate input target that simultaneously satisfies the first condition and the second condition can be selected as the final input target from the candidate input targets. Here, the first condition is the candidate input target and the information display device. The second condition is that the pixel size corresponding to the face in the target person image of the candidate input target is equal to or larger than the predetermined size, or the candidate input target's distance is equal to or less than the predetermined distance. It indicates that the ratio of the area corresponding to the face in the target person image to the reference area in the target person image is equal to or more than a predetermined ratio.

In another example, weights can be introduced to comprehensively consider the first and second factors, i.e., weights are set for each of the first and second factors, and the weights of the first factor. The value and the weight value of the second factor may be the same or different from each other. In one application scene, the information display device and the image collection device are located in a closed and narrow scene such as an elevator, and all the positions in such a scene are for the display contents of the information display device. It can be regarded as a position that forms attention, and when the face faces the information display device from the front in such a scene, the person corresponding to the face forms attention to the display content of the information display device. More likely. When the face faces the information display device from the side in such a scene, it is less likely that the person corresponding to the face has formed attention to the display content of the information display device. On the other hand, if the face faces the information display device from the front, the face size may be larger, and if the face faces the information display device from the side, the face size may be smaller. .. Therefore, in the application scene, a smaller weight value can be set for the first factor, and a larger weight value can be set for the second factor.

In another application situation, such as when the information display device and the image acquisition device are located in a wider scene such as both sides of a road, and the information display device and the image acquisition device are located in a higher place. If the size of the face image taken by the image collector in the scene is relatively small and the distance between the person corresponding to the face and the information display device is small in such a scene, the information display device displays. It is even more likely that they have formed attention to the content. Otherwise, if it is larger than the distance between the person corresponding to the face and the information display device, it may not be possible to clearly see the display content of the information display device, that is, the display content of the information display device. On the other hand, it is less likely to form attention, and therefore, in the application scene, a larger weight value can be set for the first factor and a smaller weight value can be set for the second factor.

In an actual application, when face tracking is performed on the plurality of frames of a person image, it is possible to further determine the face quality of each person image among the plurality of frames of the person image. Further, by selecting the image of one frame having the highest face quality from the images corresponding to all the candidate input targets or the images corresponding to the final input target, the image of one frame having the highest selected face quality is selected. Face recognition or subsequent processing can be performed based on. Here, the face quality of the person image of each frame can represent an index such as the sharpness of the face, the brightness of the image, the angle of the face, and whether or not the person is obstructed.

FIG. 2 is a schematic diagram of one application scene of the embodiment of the present invention, FIG. 3 is a schematic diagram of another application scene of the embodiment of the present invention, and information is displayed in FIGS. 2 and 3. The device 1 is located at a high place, and the image collecting device 2 is located directly above the information display device. Based on the above-mentioned contents of the embodiment of the present invention, the candidate input target can be determined after the processing is executed on the image collected by the image acquisition device 2, and the final input target can be determined. can. The information display device 1 can be used to display the recommendation information for the candidate input target or the final input target, and the recommendation information displayed by the information display device 1 in FIGS. 2 and 3 is a vehicle image. ..

Referring to FIG. 2, the face faces the information display device 1 from the front, and the face image collected by the image collecting device 2 is a front face image. Referring to FIG. 3, the face faces the information display device 1 from the side, and the face image collected by the image collecting device 2 is a side face image. When evaluating the face quality of an image based on the above-mentioned contents, the face quality can be evaluated based on the angle of the face. When all the other indicators (for example, the sharpness of the face, the brightness of the image, whether or not the image is obstructed) are equal, the frontal facial image collected by using the image collecting device 2 in the scene of FIG. The quality is higher than the quality of the side face image collected by using the image collecting device 2 in the scene of FIG.

Further, after selecting the one-frame image having the highest face quality, the recommendation information may be determined based on the face image of the candidate input target or the final input target corresponding to the one-frame image having the highest face quality. can.

Based on the data push method submitted by the aforementioned examples, the embodiments of the present invention propose a data push device.

FIG. 4 is a schematic diagram of the configuration of the data push device according to the embodiment of the present invention, and as shown in FIG. 4, the data push device includes an acquisition module 401, a determination module 402, and a processing module 403. You may.

The acquisition module 401 acquires a plurality of frames of human images collected by the image collection device.

The determination module 402 determines the staying time in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image.

The processing module 403 determines a candidate input target based on the stay time, determines recommendation information based on the candidate input target, and displays the recommendation information via the information display device.

When selectively determining a candidate input target based on the dwell time, the processing module 403 specifically

It is configured to target a person corresponding to a face having a staying time of a predetermined time or longer and / or a person corresponding to a face having a staying time within a predetermined range as a candidate input target.

Optionally, the imaging range of the image acquisition device includes the front area of the information display device.
The image collecting device is deployed on the information display device, or the distance between the image collecting device and the information display device is equal to or less than a predetermined distance threshold value.

When selectively determining the recommendation information based on the candidate input target, the processing module 403 specifically
Based on the distance between each candidate input target among the candidate input targets and the information display device, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is determined. The distance between the information display device and the information display device is equal to or less than a predetermined distance.
It is configured to determine the recommendation information for the final input target.

When selectively determining the recommendation information based on the candidate input target, the processing module 403 specifically
Based on the face size of each candidate input target among the candidate input targets in the target person image, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is said to be the same. The pixel size corresponding to the face in the target person image is equal to or larger than a predetermined size, or the ratio of the area corresponding to the face in the target person image of the final input target to the reference area in the target person image is determined. It is above the specified ratio and

It is configured to determine the recommendation information for the final input target.

Optionally, the processing module 403 further determines the predetermined size based on the image acquisition resolution of the image acquisition device, which positively correlates with the image acquisition resolution of the image acquisition device. It is configured to have.

When selectively determining the time spent in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image, the determination module 402 specifically comprises.
By performing face tracking on the plurality of frames of the person image, it is configured to determine the time of stay of the face in the plurality of frames of the person image in front of the information display device.

In the actual application, the acquisition module 401, the determination module 402, and the processing module 403 can all be realized by utilizing the processors in the electronic device, and the above-mentioned processors are ASICs, DSPs, DSPDs, and PLDs. , FPGA, CPU, controller, microcontroller, microprocessor can be at least one.

Further, each functional module of this embodiment may be integrated into one processing unit, each unit may be physically present independently, or two or more units may be one unit. It may be integrated into. The integrated unit described above may be implemented in the form of using hardware or in the form of using software function modules.

If the integrated unit is realized in the form of software functional modules without being sold or used as a stand-alone product, it may be stored on one computer-readable storage medium. Based on this understanding, the examples may be embodied in the form of software products in nature, or all or part of the examples may be embodied in the form of software products. can. The computer software product is stored in one storage medium, and one computer device (which may be a personal computer, server, network device, etc.) or processor (processor) is all or one of the methods described in this embodiment. Includes some instructions to perform a part step. The above-mentioned storage medium is various media that can store program codes such as U disk, mobile hard disk, read-only memory (Read Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk, and optical disk. including.

Specifically, the computer program instruction corresponding to the data push method of the present embodiment can be stored in a storage medium such as an optical disk, a hard disk, or a U disk, and the computer program instruction corresponding to the data push method in the storage medium can be stored. When read or executed by an electronic device, any data push method of the above-described embodiment is realized.

Based on the embodiments described above, with reference to FIG. 5, FIG. 5 shows an electronic device 500 provided by an embodiment of the invention, which electronic device 500 may include a memory 501 and a processor 502.
The memory 501 stores a computer program and data, and stores the computer program and data.
The processor 502 realizes any data pushing method of the above-described embodiment by executing a computer program stored in the memory.

In actual application, the above memory 501 includes a volatile memory such as RAM, a ROM, a flash memory, a hard disk (Hard Disk Drive, HDD), a solid state disk (Solid-State Drive, SSD). ) Or a non-volatile memory, or a combination of the above types of memory, providing instructions and data to the processor 502.

The processor 502 may be at least one of an ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It can be understood that the electronic devices may be yet other to realize the processor functions described above by different extended reality cloud platforms, and the embodiments of the present invention are not specifically limited.

The embodiments of the present invention further provide a computer program, and when the computer program is executed by a processor, any data pushing method of the above-described embodiment is realized.

In some embodiments, the functionality provided by the apparatus provided by the embodiments of the present invention or the modules included in the apparatus may be used to perform the methods described in the embodiments of the above methods. However, the specific realization thereof may be described by referring to the description of the embodiment of the above method, and is not described repeatedly here for the sake of simplification.

The description for each example above tends to emphasize the differences between the examples, and the same or similar points may be referred to each other and will not be repeated here for the sake of brevity.

The methods disclosed by the examples of each method provided by the present invention can be any combination without conflict to obtain new product examples.

The features disclosed by each of the products provided by the present invention can be arbitrarily combined to obtain new product embodiments without conflict.

The features disclosed by the embodiments of each method or device provided by the present invention can be arbitrarily combined without conflict to obtain an embodiment of a new method or an embodiment of a device.

Those skilled in the art can implement the embodiment of the above method by the method of software and the required general-purpose hardware platform, and of course, by hardware, by describing the above embodiments. However, in many cases it can be clearly understood that the former is a better embodiment. Based on this understanding, the embodiments of the present invention can be embodied essentially in the form of software products, wherein the computer software product is one storage medium (eg, ROM / RAM, magnetic disk). , Optical disk), and several instructions so that one terminal (mobile phone, computer, server, air conditioner, network device, etc.) can perform the method described in each embodiment of the present invention. including.

Although the embodiments of the present invention have been described above with reference to the drawings, the present invention is not limited to the above-mentioned specific embodiments. The above-mentioned specific embodiment is merely schematic and is not limited. Those skilled in the art can design many embodiments under the suggestion of the present invention without departing from the scope of the intent and claims of the present invention, all of which are the scope of protection of the present invention. Belongs to.

Claims (21)

It ’s a data push method.
A step to acquire a multi-frame portrait image collected by an image collector, and
A step of determining the time spent in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image, and
The step of determining the candidate input target based on the staying time and
A data push method comprising: determining recommendation information based on the candidate input target and displaying the recommendation information via the information display device. The step of determining the candidate input target based on the staying time is
It is characterized by including a step of targeting a person corresponding to a face having a staying time of a predetermined time or more and / or a person corresponding to a face having a staying time within a predetermined range as a candidate input target. The data push method according to claim 1. The photographing range of the image collecting device includes the front area of the information display device, and includes the front area of the information display device.
The image collecting device is provided on the information display device, or the distance between the image collecting device and the information display device is equal to or less than a predetermined distance threshold. Or the data push method according to 2. The step of determining recommendation information based on the candidate input target is
It is a step of determining at least one candidate input target among the candidate input targets as the final input target based on the distance between each candidate input target among the candidate input targets and the information display device. The step that the distance between the final input target and the information display device is equal to or less than a predetermined distance,
The data push method according to any one of claims 1 to 3, wherein the step of determining the recommendation information for the final input target is included. The step of determining recommendation information based on the candidate input target is
This is a step of determining at least one candidate input target among the candidate input targets as the final input target based on the face size of each candidate input target in the target person image. The pixel size corresponding to the face in the target person image of the input target is equal to or larger than a predetermined size, or the reference area in the target person image of the area corresponding to the face in the target person image of the final input target. Steps where the ratio to is greater than or equal to the predetermined ratio,
The data push method according to any one of claims 1 to 3, wherein the step of determining the recommendation information for the final input target is included. A step of determining the predetermined size based on the image acquisition resolution of the image acquisition device, wherein the predetermined size further includes a step having a positive correlation with the image acquisition resolution. The data push method according to claim 5. Based on the multi-frame person image, the step of determining the staying time in front of the face information display device in the multi-frame person image is a step.
1. The first aspect of the present invention includes a step of determining the staying time of a face in the plurality of frames of a person image in front of the information display device by performing face tracking on the plurality of frames of the person image. The data push method according to any one of 6 to 6. The step of determining the face quality in each person image among the multiple frames of the person image, and
The data push method according to claim 7, further comprising a step of selecting a one-frame image having the highest face quality from the images corresponding to the candidate input target. The step of determining recommendation information based on the candidate input target is
The data push method according to claim 8, further comprising a step of determining recommendation information based on the face image of the candidate input target corresponding to the one-frame image having the highest face quality. It ’s a data push device,
An acquisition module for acquiring multiple frames of human images collected by an image collection device,
A determination module for determining the time spent in front of the face information display device in the plurality of frames of the person image based on the plurality of frames of the person image, and a determination module.
It is characterized by comprising a processing module for determining a candidate input target based on the staying time, determining recommendation information based on the candidate input target, and displaying the recommendation information via the information display device. The data push device according to claim. When determining a candidate input target based on the dwell time, the processing module
It is configured so that a person corresponding to a face whose stay time is longer than a predetermined time and / or a person corresponding to a face whose stay time is within a predetermined range is set as a candidate input target. The data push device according to claim 10. The photographing range of the image collecting device includes the front area of the information display device, and includes the front area of the information display device.
10. The image collecting device is provided on the information display device, or the distance between the image collecting device and the information display device is equal to or less than a predetermined distance threshold. Or the data push device according to 11. When determining recommendation information based on the candidate input target, the processing module specifically
Based on the distance between each candidate input target among the candidate input targets and the information display device, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is determined. The distance between the information display device and the information display device is equal to or less than a predetermined distance.
The data push device according to any one of claims 10 to 12, wherein the recommendation information is determined with respect to the final input target. When determining recommendation information based on the candidate input target, the processing module specifically
Based on the face size of each candidate input target among the candidate input targets in the target person image, at least one candidate input target among the candidate input targets is determined as the final input target, and the final input target is said to be the same. The pixel size corresponding to the face in the target person image is equal to or larger than a predetermined size, or the ratio of the area corresponding to the face in the target person image of the final input target to the reference area in the target person image is determined. It is above the specified ratio and
The data push device according to any one of claims 10 to 12, wherein the recommendation information is determined with respect to the final input target. The processing module further determines the predetermined size based on the image acquisition resolution of the image acquisition device, and the predetermined size is configured to have a positive correlation with the image acquisition resolution. 14. The data push device according to claim 14. When determining the time spent in front of the face information display device in the multi-frame person image based on the multi-frame person image, the determination module specifically comprises.
It is characterized in that it is configured to determine the staying time of a face in the plurality of frames of a person image in front of the information display device by performing face tracking on the plurality of frames of the person image. The data push device according to any one of claims 10 to 15. The determination module further
The face quality in each person image among the multiple frames of person images is determined.
The data push device according to claim 16, wherein one frame image having the highest face quality is selected from the images corresponding to the candidate input target. The processing module is specifically configured to determine recommendation information based on the face image of the candidate input target corresponding to the one-frame image having the highest face quality. 17. The data push device according to 17. It ’s an electronic device,
With the processor
It is equipped with a memory for storing a computer program that can be executed by the processor.
An electronic device according to claim 1, wherein the data push method according to any one of claims 1 to 9 is executed when the computer program is executed by the processor. A computer storage medium in which computer programs are stored.
A computer storage medium, characterized in that, when the computer program is executed by a processor, the processor implements the data push method according to any one of claims 1 to 9. It ’s a computer program,
A computer program, characterized in that, when the computer program is executed by a processor, the processor implements the data push method according to any one of claims 1 to 9.

Images