CN113724454B

CN113724454B - Interaction method of mobile equipment, device and storage medium

Info

Publication number: CN113724454B
Application number: CN202110980245.5A
Authority: CN
Inventors: 郭双; 方林; 唐旋来
Original assignee: Shanghai Keenlon Intelligent Technology Co Ltd
Current assignee: Shanghai Keenlon Intelligent Technology Co Ltd
Priority date: 2021-08-25
Filing date: 2021-08-25
Publication date: 2022-11-25
Anticipated expiration: 2041-08-25
Also published as: CN113724454A

Abstract

The application discloses an interaction method of a mobile device, the mobile device, a device and a storage medium, relates to the technical field of artificial intelligence, and can enrich the functions of the mobile device and improve the interaction effect of the mobile device and a user. The method comprises the following steps: determining that a moving object appears in a first preset area, wherein the first preset area is larger than a preset service area; acquiring an image corresponding to a moving object, identifying the image corresponding to the moving object, and judging whether a moving person exists or not; if yes, broadcasting according to the propaganda content corresponding to the current activity scene; and under the condition that the mobile personnel enter the preset service area, determining interactive content according to the activity information corresponding to the current activity scene and/or the user attribute of the mobile personnel, and controlling the mobile equipment to interact with the mobile personnel according to the interactive content.

Description

Interaction method of mobile equipment, device and storage medium

Technical Field

The embodiment of the application relates to the technical field of artificial intelligence, in particular to an interaction method of mobile equipment, the mobile equipment, a device and a storage medium.

Background

With the rapid development of artificial intelligence technology, robots are more and more widely applied. For example, the robot can be placed at the door of a place such as a mall to welcome a guest instead of a human. At present, a method for a robot to interact with a user in a welcome mode is that when the robot recognizes that an object moves according to a sensor, a preset welcome language is played.

However, in the above method for the interaction between the robot and the user, the function of the robot is too single, which affects the interaction effect with the user, so a new method for the interaction between the robot and the user is needed to be proposed to improve the interaction effect between the existing robot and the user.

Disclosure of Invention

The application provides an interaction method, a mobile device, a device and a storage medium of the mobile device, which can enrich the functions of the mobile device and improve the interaction effect of the mobile device and a user.

In order to achieve the purpose, the technical scheme is as follows:

in a first aspect, the present application provides an interaction method for a mobile device, including: determining that a moving object appears in a first preset area, wherein the first preset area is larger than a preset service area; acquiring an image corresponding to a moving object, identifying the image corresponding to the moving object, and judging whether a moving person exists or not; if yes, broadcasting according to the propaganda content corresponding to the current activity scene; and under the condition that the mobile personnel enter the preset service area, determining interactive content according to the activity information corresponding to the current activity scene and/or the user attribute of the mobile personnel, and controlling the mobile equipment to interact with the mobile personnel according to the interactive content.

In a second aspect, the present application provides a mobile device, including a detection module, an acquisition module, a broadcast module, and an interaction module; the mobile terminal comprises a detection module, a processing module and a processing module, wherein the detection module is used for detecting and determining a mobile object in a first preset area, and the first preset area is larger than a preset service area; the acquisition module is used for acquiring an image corresponding to the moving object, identifying the image corresponding to the moving object through the identification module and judging whether a moving person exists or not; the broadcasting module is used for broadcasting according to the propaganda content corresponding to the current activity scene under the condition that the identification module determines that the mobile personnel exist; and the interaction module is used for determining interaction content according to the activity information corresponding to the current activity scene and/or the user attribute of the mobile personnel under the condition that the mobile personnel enter the preset service area, and controlling the mobile equipment to interact with the mobile personnel according to the interaction content.

In a third aspect, the present application provides an interaction apparatus for a mobile device, including a memory, a processor, a bus, and a communication interface; the memory is used for storing computer execution instructions, and the processor is connected with the memory through a bus; when the interaction means of the mobile device is operated, the processor executes the computer-executable instructions stored in the memory, so that the interaction means of the mobile device performs the interaction method of the mobile device as provided in the first aspect.

Optionally, the interaction means of the mobile device may further comprise a transceiver for performing the steps of transceiving data, signaling or information, for example, acquiring target images of at least two moving persons, under the control of the processor of the interaction means of the mobile device.

Further optionally, the interaction device of the mobile device may be a physical machine for implementing interaction of the mobile device, or may be a part of the physical machine, for example, a system on chip in the physical machine. The chip system is used for supporting the interaction means of the mobile device to implement the functions referred to in the first aspect, for example, to receive, transmit or process data and/or information referred to in the interaction method of the mobile device. The chip system includes a chip and may also include other discrete devices or circuit structures.

In a fourth aspect, the present application provides a computer-readable storage medium, in which instructions are stored, and when the instructions are executed by a computer, the computer is enabled to execute the interaction method of the mobile device as provided in the first aspect.

In a fifth aspect, the present application provides a computer program product comprising computer instructions which, when run on a computer, cause the computer to perform the method of interaction of a mobile device as provided in the first aspect.

It should be noted that all or part of the computer instructions may be stored on the computer readable storage medium. The computer-readable storage medium may be packaged with a processor of the interaction apparatus of the mobile device, or may be packaged separately from the processor of the interaction apparatus of the mobile device, which is not limited in this application.

In the present application, the names of the interaction means of the mobile device mentioned above do not limit the devices or functional modules themselves, which may appear by other names in practical implementations. Insofar as the functions of the respective devices or functional modules are similar to those of the present application, they fall within the scope of the claims of the present application and their equivalents.

According to the technical scheme, before interaction, the image corresponding to the moving object is identified, whether moving personnel exist is judged, and the propaganda content corresponding to the current activity scene is broadcasted under the condition that the moving personnel exist. Therefore, the interactive object of the mobile equipment can be ensured to be a person, the mobile equipment is prevented from carrying out invalid interaction with other objects, and the interactive effect is improved. In addition, the broadcasting method and the broadcasting device are used for broadcasting after the mobile object is determined to appear in the first preset area and the mobile personnel are determined to exist in the first preset area, and the first preset area is larger than the preset service area of the mobile device. Therefore, the mobile personnel outside the preset service area can be attracted to the preset service area to interact with the mobile equipment in a close range through broadcasting the propaganda content, and the interaction effect of the mobile equipment and a user can be further improved. In addition, after the mobile personnel enter the preset service area, the personalized interactive content meeting the user requirements can be determined according to the activity information corresponding to the current activity scene and/or the user attributes of the mobile personnel, so that the interactive effect of the mobile equipment and the user is further improved, and the user experience is improved. It can be seen that, in the technical scheme provided by the application, the interaction effect between the mobile device and the user can be improved by enriching the functions of the mobile device (determining that the interaction object is a mobile person, advertising and broadcasting the mobile person to a preset service area, and determining personalized interaction content).

These and other aspects of the present application will be more readily apparent from the following description.

Drawings

Fig. 1 is a schematic flowchart illustrating an interaction method of a mobile device according to an embodiment of the present disclosure;

fig. 2 is a flowchart illustrating another interaction method of a mobile device according to an embodiment of the present disclosure;

fig. 3 is a schematic flowchart illustrating another interaction method of a mobile device according to an embodiment of the present disclosure;

fig. 4 is a schematic structural diagram of an interaction apparatus of a mobile device according to an embodiment of the present disclosure;

fig. 5 is a schematic structural diagram of another interaction apparatus for a mobile device according to an embodiment of the present disclosure.

Detailed Description

The following describes in detail an interaction method of a mobile device, an apparatus and a storage medium provided in an embodiment of the present application with reference to the accompanying drawings.

The term "and/or" herein is merely an association describing an associated object, meaning that three relationships may exist, e.g., a and/or B, may mean: a exists alone, A and B exist simultaneously, and B exists alone.

The terms "first" and "second" and the like in the description and drawings of the present application are used for distinguishing different objects or for distinguishing different processes for the same object, and are not used for describing a specific order of the objects.

Furthermore, the terms "including" and "having," and any variations thereof, as referred to in the description of the present application, are intended to cover non-exclusive inclusions. For example, a process, method, system, article, or apparatus that comprises a list of steps or elements is not limited to only those steps or elements listed, but may alternatively include other steps or elements not listed, or inherent to such process, method, article, or apparatus.

It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described herein as "exemplary" or "e.g.," is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "such as" is intended to present concepts related in a concrete fashion.

In the description of the present application, the meaning of "a plurality" means two or more unless otherwise specified.

In view of the above problems in the prior art, embodiments of the present application provide an interaction method for a mobile device, which can improve an interaction effect between the mobile device and a user by enriching functions of the mobile device (determining that an interaction object is a mobile person, and determining personalized interaction content by using a broadcast to attract the mobile person to a preset service area).

The interaction method of the mobile equipment provided by the embodiment of the application can be applied to a mobile equipment body or a chip system in the mobile equipment. Wherein the mobile device may be a robot. Of course, in practical applications, the mobile device may also be other movable artificial intelligence devices.

The interaction method of the mobile device provided in the embodiment of the present application may also be applied to an interaction device of the mobile device, where the interaction device of the mobile device may be a physical machine (for example, a server) for implementing interaction between the mobile device and a mobile person, or may be a part of a device in the physical machine, for example, a chip system in the physical machine.

In the following description of the embodiment of the present application, an execution main body of the interaction method of the mobile device is taken as an example of a mobile device body for expansion and description.

The application scenario of the interaction method of the mobile device provided in the embodiment of the present application may be a guest greeting scenario of the robot, and certainly, in practical application, the application scenario may also be applied to other application scenarios, which is not limited in the embodiment of the present application. The interaction method of the mobile device provided by the embodiment of the present application will be described in detail below by taking a guest greeting scene as an example.

Referring to fig. 1, the interaction method of the mobile device provided in the embodiment of the present application includes S101-S104:

s101, the mobile device determines that a moving object appears in a first preset area.

The mobile device can be provided with a distance measuring sub-device such as a laser radar and/or a binocular stereoscopic vision camera, and the distance measuring sub-device can detect the distance between the mobile device and an object in a first preset area. For example, taking the laser radar to perform distance detection as an example, the laser radar may acquire a point cloud image in a first preset area, an object in the first preset area is in a shape of a point cluster in the point cloud image, and the laser radar may measure a distance between a center of the point cluster in the point cloud image and the mobile device. The mobile device can determine whether the object corresponding to the center of the point cluster is a moving object according to whether the distance between the center of each point cluster detected by the laser radar and the mobile device changes.

The predetermined service area may be determined according to a service range of the mobile device, and may be, for example, an area within 2 meters of the mobile device.

The first preset area may be an area determined in advance to be larger than a preset service area. Illustratively, the first predetermined area may be an area within 20 meters of the surroundings of the mobile device.

Optionally, in a possible implementation manner, in order to expand the range of detecting the moving object as much as possible and ensure that the moving object that may exist around the mobile device may be detected, the ranging sub-device may be disposed circumferentially around the mobile device. In this way, moving objects within a 360 degree field of view of the mobile device may be detected.

Optionally, the mobile device may detect a moving object in a detection area of the mobile device, and if it is detected that the moving object appears in a plurality of sub-areas in the detection area, determine a sub-area meeting a preset angle condition in the plurality of sub-areas as a first preset area.

The detection view of the mobile device is large, for example, when the mobile device with the detection view of 360 degrees is applied to an application scene of a shopping mall, if the shopping mall is in a busy period, the number of the mobile objects detected by the mobile device is large. Therefore, in order to better adapt to the requirements of different application scenes and improve the interaction efficiency, the mobile device can detect the mobile object according to the predetermined angle range, so that the interactive object meeting the requirements of the application scenes can be quickly screened out.

The detection area may be a field of view area of the mobile device, and the detection area may be divided into a plurality of sub-areas according to a field of view angle in advance. Illustratively, the detection region is a region within 360 degrees of the field of view of the mobile device, one subregion for every 30 degrees.

The preset angle condition may be a condition determined in advance according to the requirements of the application scenario. Illustratively, if the application scenario requires that a new customer is attracted to a certain market as much as possible, the mobile device interacts with the new customer to publicize the market. The preset angle condition may be a direction of 30 degrees around the front entrance into the mall gate, which is a general angle at which a new customer enters the mall. Like this, when moving object appears in a plurality of subregion in the detection area, can preferentially detect the subregion that new customer probably appears to make mobile device interact with the customer when new customer appears, promote the effect of publicity.

S102, the mobile device acquires an image corresponding to the moving object, identifies the image corresponding to the moving object and judges whether a moving person exists or not.

The application scenes of the mobile device are many, and in a case where the surrounding environment of the application scene is complicated, the mobile object is not necessarily a human being, and may be another mobile device, or a movable object such as an animal. In order to ensure that an interactive object of the mobile device is a person, avoid invalid interaction between the mobile device and other objects, and improve an interactive effect, after the mobile device determines that the mobile object appears in the first preset area according to the distance measuring sub-device, the mobile device can start a collecting sub-device (such as a camera) arranged on the mobile device to collect an image corresponding to the mobile object, identify the collected image, and judge whether a mobile person exists. Wherein the image corresponding to the moving object may be an image including the moving object. In addition, the mobile object is determined by the distance measuring sub-equipment (such as a laser radar) firstly, and then the collecting sub-equipment (such as a binocular stereoscopic vision camera) is started to collect images, so that the problem of overlarge data processing amount caused by simultaneous processing of various types of data can be avoided, and the power consumption of the general collecting sub-equipment and the data processing amount of the collected data are large, so that the resource waste caused by the fact that the collecting sub-equipment is always in a working state can be avoided.

Optionally, in a possible implementation manner, after the mobile device acquires the image corresponding to the moving object, the mobile device may perform face feature recognition on the image, and determine whether there is a moving person according to a result of the face feature recognition. For example, the mobile device may perform face recognition on a moving object in the image according to a face recognition algorithm such as an eigenface algorithm, an elastic bundle map matching algorithm, a flexible appearance model algorithm, or a deep learning algorithm, to determine whether the moving object is a moving person. It can be understood that, in practical application, the face recognition of the moving object in the image may also be performed by using other face recognition algorithms, which is not limited in this application.

Optionally, in another possible implementation manner, after the mobile device determines that the moving object appears in the first preset area according to the distance measuring sub-device, an infrared detector arranged on the mobile device may be started, and whether a moving person appears is determined according to infrared imaging detected by the infrared detector. Of course, in practical application, the mobile personnel can be detected in other manners, and the embodiment of the present application provides only a few preferred detection manners, and does not constitute a limitation on the manner of detecting the mobile personnel.

Optionally, the mobile device may control the body of the mobile device or the acquisition sub-device of the mobile device to rotate according to the position of the moving object, so as to obtain an image corresponding to the moving object.

Because the range finding sub-equipment and the collection sub-equipment that set up on the mobile device can be different equipment, for example, the range finding sub-equipment can be lidar, and the collection sub-equipment can be the camera. The acquisition fields of view may be different for different devices at different positions, and thus, when the mobile device detects a moving object in the first preset area according to the distance measured by the ranging sub-device, the moving object may be located outside the acquisition field of view of the acquisition sub-device. Therefore, in order to ensure that the acquisition sub-device acquires the image including the moving object, the mobile device may control the body of the mobile device or the acquisition sub-device of the mobile device to rotate according to the position of the moving object until the image corresponding to the moving object is acquired.

Generally, the rotation speed of the acquisition sub-device is faster than that of the body of the mobile device, and in order to quickly acquire an image corresponding to a moving object, optionally, the acquisition range of the acquisition sub-device may be expanded as much as possible, and the rotation of the body of the mobile device may be reduced. Illustratively, the acquisition sub-device may be configured to be up and down 180 degrees and rotatable left and right 180 degrees.

Optionally, in order to meet the requirements of different application scenarios, the acquisition range of the acquisition sub-device may be adjusted according to the requirements of the application scenarios. For example, when it is required to preferentially capture an image of a mobile person in a field of view directly in front of the mobile device, the capture sub-device may be configured to be rotatable by 60 degrees left and right.

And S103, broadcasting according to the propaganda content corresponding to the current activity scene when the mobile device determines that the mobile personnel exist.

The promotional content corresponding to the current campaign scenario may be predetermined based on the campaign scenario. For example, taking the application scenario as a shopping mall, the activity scenario may be daily promotion, festival promotion, business promotion, or yearly celebration. Correspondingly, the propaganda content can be the propaganda oral number etc. of confirming according to the activity scene in advance, broadcasts through these propaganda content, can promote the attraction effect to the removal personnel. In addition, when the mobile device determines that mobile personnel exist, voice broadcast and/or video broadcast can be carried out according to the propaganda content corresponding to the current activity scene, so that the problem that the attraction effect is poor and the like caused by single fixed propaganda content can be avoided.

In the embodiment of the application, the mobile device can broadcast the propaganda content after determining that the mobile personnel exist in the first preset area. Can attract the mobile personnel who predetermine the service area outside to predetermineeing the service area in through the propaganda and broadcast and closely interdynamic with the mobile device to can improve mobile device and user's interactive effect.

S104, under the condition that the mobile device determines that the mobile personnel enter the preset service area, the mobile device determines interactive content according to activity information corresponding to the current activity scene and/or user attributes of the mobile personnel, and controls the mobile device to interact with the mobile personnel according to the interactive content.

After the mobile personnel enter the preset service area, the mobile equipment can determine the personalized interactive content meeting the user requirements according to the activity information corresponding to the current activity scene and/or the user attributes of the mobile personnel, so that the interactive effect between the mobile equipment and the user can be further improved, and the user experience is improved.

Optionally, after the mobile device broadcasts the promotion content corresponding to the current activity scene, whether the mobile personnel enter a second preset area can be judged; under the condition that the mobile personnel are determined to enter the second preset area, the mobile equipment can perform target tracking on the mobile personnel according to the continuous frame images of the mobile personnel, and then determine whether the mobile personnel enter the preset service area according to the tracking data acquired in the target tracking process.

According to the technical scheme, the target can be locked in stages, the mobile object is locked first, then the mobile personnel in the mobile object are locked, and after the mobile personnel are locked, the mobile personnel can be tracked to determine whether the mobile personnel enter the preset service area. Generally, in order to attract objects outside a preset service area as much as possible, the range of the first preset area is set to be large, so that the distance between the mobile personnel and the mobile equipment is possibly far, the target tracking is inconvenient to perform, and wrong objects can be tracked in the tracking process. Therefore, a second predetermined area may be preset, which is smaller than the first predetermined area and larger than the predetermined service area. When the mobile personnel enter a second preset area, the mobile personnel are tracked by the target, so that errors in the tracking process can be avoided.

Wherein the second preset area may be determined in advance according to the farthest tracking distance of the mobile device. Illustratively, the second predetermined area may be an area within 10 meters of the surroundings of the mobile device.

After the mobile device determines that the mobile person enters the second preset area, the acquisition sub-device can be started to acquire continuous frame images of the mobile person, and target tracking is performed on the mobile person according to the continuous frame images. In the target tracking process, the mobile device may determine whether the mobile person enters a preset service area according to the acquired tracking data. Wherein the tracking data may be image features of a moving person in successive frames of images. Illustratively, the tracking data may be a background color (e.g., a clothing color) or a texture feature (e.g., a clothing texture) of the moving person in the successive frame images. Certainly, in practical applications, the mobile device may also perform target tracking on the mobile person according to other tracking data, which is not limited in this embodiment of the present application.

Optionally, the mobile device may obtain continuous frame images of the mobile person according to a preset frequency; and then adjusting acquisition parameters according to the difference of at least two frames of images in the continuous frames of images, and tracking the target of the moving person.

Because the mobile personnel is in a moving state, if the mobile personnel moves out of the acquisition field of view of the acquisition sub-equipment of the mobile equipment in the process of acquiring the continuous frame images of the mobile personnel by the mobile equipment, the mobile equipment cannot acquire the images of the mobile personnel, and the target tracking of the mobile personnel cannot be performed. Therefore, to further ensure successful tracking of the mobile person, in acquiring the successive frames of images of the mobile person, the acquisition parameters may be adjusted according to a difference between at least two frames of images, by adjusting the acquisition parameters to ensure that the mobile person is always within the acquisition field of view of the acquisition sub-device of the mobile device.

The preset frequency may be a frequency at which the image is acquired by the acquisition sub-apparatus determined in advance. For example, the preset frequency may be 5 times per second, that is, the acquiring sub-device acquires 5 images per second. The acquisition parameters may include parameters such as an up-down rotation angle, a left-right rotation angle, and a focal length of the acquisition sub-device.

The difference of the at least two images may be a difference in position of the moving person in the at least two images. For example, the mobile device may predict a movement trend of the mobile person according to a position change trend of the mobile person in at least two frames of images, and adjust the acquisition parameters of the acquisition sub-device according to the movement trend of the mobile person, so as to ensure that the mobile person is always within an acquisition range of the acquisition sub-device.

Optionally, before determining whether the mobile personnel enters the preset service area, the mobile device may further determine whether the mobile personnel enters a third preset area according to the tracking data obtained in the target tracking process; and under the condition that the mobile personnel is determined to enter the third preset area, the mobile equipment performs feature recognition on the continuous frame images and determines the user attribute of the mobile personnel.

In order to enrich the functions of the mobile device and improve the interaction effect, in the technical scheme provided by the embodiment of the application, the user attribute of the mobile personnel can be determined according to the result of feature recognition by carrying out feature recognition on the continuous frame images of the mobile personnel, and then the personalized interaction content meeting the user requirement can be determined according to the user attribute of the mobile personnel. In order to enable the mobile device to interact with the mobile personnel according to the interaction content once the mobile personnel enter the preset service area, the interaction efficiency is improved, and the user attributes of the mobile personnel can be identified before the mobile personnel enter the preset service area. Therefore, a third preset area can be preset, the third preset area is smaller than the second preset area and larger than the preset service area, and the mobile personnel can start to identify the user attributes when entering the third preset area, so that the time before the mobile equipment enters the preset service area can be better utilized, and the user experience is improved.

The third preset area may be an area determined in advance. Illustratively, the third predetermined area may be an area within 5 meters of the surroundings of the mobile device.

The user attributes may include at least one of gender, age, height, historical consumption information, and expected consumption levels.

Since preferences of users of different genders, different ages, and different heights may differ, user attributes may include one or more of gender, age, or height. For example, the mobile device may invoke a pre-trained deep learning based age detector to perform feature recognition on successive frame images of the mobile person to determine the age of the mobile person. For another example, the mobile device may invoke a pre-trained deep learning based height classifier to perform feature recognition on successive frame images of the mobile person to determine the height of the mobile person. For another example, the mobile device may perform feature recognition on the continuous frame images of the mobile person according to a gender recognition algorithm based on human face features, a gender recognition method based on Fisher's criterion, or a gender classification algorithm based on an adaptive boost and support vector machine (Adaboost + SVM), to determine the gender of the mobile person. It is understood that in practical applications, the gender, age or height of the moving person may be determined by performing feature recognition on the continuous frame images in other manners, which is not limited in the embodiment of the present application.

In addition, the historical consumption information of the user reflects the historical consumption condition of the user, the historical consumption condition can represent the preference of the user, and therefore, the user attribute can also comprise the historical consumption information.

For example, the historical consumption information may include the type of the purchased product, information on the visited stores (the time of visit, the number of visits, etc.), and the like. The historical consumption information and the corresponding face feature information can be stored in a database of the mobile device or a background server of the mobile device, and after the mobile device locks the mobile personnel according to the continuous frame images, the face features of the mobile personnel can be subjected to feature matching with the faces in the database, so that the historical consumption information corresponding to the mobile personnel is determined. If the matching of the face features of the mobile personnel and the face features in the database fails, the face features of the mobile personnel can be stored in the database, and the consumption information of the mobile personnel can be conveniently acquired subsequently.

The expected consumption level may be a consumption level corresponding to the historical consumption amount predicted from the correspondence of the consumption amount to the consumption level according to the historical consumption amount. The historical consumption amount can reflect the shopping level of a user, the expected consumption level can predict the shopping level of the user, and the goods recommended to the user according to the expected consumption level can better meet the requirements of the user, so the user attribute can also comprise the expected consumption level.

In another possible implementation, the expected consumption level may be determined based on clothing information (clothing, watches, bags, etc.) of a moving person in successive frame images, or the like. For example, the population with more accessories often has a higher consumption level, and the expected consumption level can be determined according to the type, the number and the like of the accessories, so that personalized recommendation can be carried out based on the expected consumption level. The type and the number of the accessories can be obtained by processing the image according to a preset classifier.

Optionally, the user attribute may include a group category, and after determining that the current time period is in a non-peak period and broadcasting according to the promotional content corresponding to the current activity scene, the method further includes: determining that at least two mobile personnel enter a second preset area; acquiring target images of at least two mobile personnel, and extracting group characteristic information of the at least two mobile personnel in the target images; judging whether at least two mobile personnel belong to the same group according to the group characteristic information; and if so, determining the group category to which at least two mobile persons belong.

When a mobile person enters an activity scene, the mobile person is mostly accompanied by a group, and the demands and preferences of different groups may be different. Therefore, the group classification can be carried out on the mobile personnel entering the second preset area, and the personalized interactive content can be determined according to the classification result.

The distance between the mobile persons in the same group is generally very close, so the group characteristic information can be the distance between the mobile persons, that is, whether the mobile persons are in the same group can be determined according to the distance between the mobile persons. For example, after determining that at least two moving persons enter the second preset area, the mobile device may acquire a target image including the at least two moving persons, extract a distance between the at least two moving persons in the target image, and if the distance in the continuous target images is less than 60cm, determine that the at least two moving persons belong to the same group. For example, the distance of the at least two moving persons may be the pupillary distance of the at least two moving persons.

The group categories may include parent-child families, girlfriends, couples, and the like. The number, sex, height and distance of the mobile personnel in different groups are generally different, and the group types of at least two mobile personnel can be determined according to the number, sex, height and distance. For example, when three mobile persons are determined according to the target image and include two adults and a child of different genders, the group category to which the three mobile persons belong may be determined as a family; when two moving persons with different sexes are determined according to the target image and the distance is less than 30cm, the group categories to which the two moving persons belong can be determined as couples; when it is determined that there are two moving persons, both of which are women, according to the target image, the group category to which the two moving persons belong may be determined to be a girlfriend.

In order to ensure the recognition effect of the group characteristics, the mobile device may recognize the group characteristics when it is determined that the current time period is in the non-peak period. For example, the rush hour period may be determined in advance based on historical people flow data, and then it may be determined whether the current period belongs to the rush hour period. Or, the mobile device may also determine people stream data of the current time period according to the real-time image acquired by the acquisition sub-device, and when the data exceeds a preset data threshold, the current time period may be determined to be a peak period.

Optionally, the mobile device may determine, from an interactive content library, interactive content corresponding to a group category to which at least two mobile people belong, where the interactive content library includes a correspondence between the group category and the interactive content.

Generally, the needs and preferences of different group categories are different, for example, family members generally search for children's amusement parks first when entering an activity scene, a shop frequently entered by girlfrieds is a female shopping shop or a snack shop, lovers are cinemas or restaurants suitable for appointments, and a multi-person college party goes to a restaurant with a compartment capable of accommodating a large number of people. Therefore, in order to determine the personalized interactive content meeting the requirements for different groups more efficiently, the interactive content belonging to each group category can be determined in advance according to each group category, and then the corresponding relation between the group category and the interactive content is stored in the interactive content library, so that the subsequent recommendation is facilitated.

Optionally, in response to a trigger instruction, determining interactive content according to the activity information, where the trigger instruction is triggered according to a trigger operation of a mobile person or a voice password; or, responding to the trigger instruction, and determining interactive content according to the user attribute; or responding to the trigger instruction, screening target activity information from the activity information according to the user attribute, and determining interactive content according to the target activity information.

In order to meet the requirements of different application scenes and improve user experience so as to further enrich the functions of the mobile device, the mobile device can determine interactive content according to the activity information of the current activity scene. Further, in order to improve the recommendation success rate of recommending the activity information of the current activity scene to the mobile personnel, the interactive content may be determined by combining the user attribute and the activity information of the mobile personnel.

The triggering operation can be an operation of inputting, clicking, sliding or long pressing on a touch screen of the mobile device by a mobile person. For example, when the mobile person clicks the "offer information" on the touch screen, the interactive content may be to display all the offer information in the current shopping mall, or to display the offer information in the current shopping mall that matches the user attribute of the mobile person. For another example, when the mobile person inputs "acquire an electronic parking ticket" in the input frame of the query bar on the touch screen, the interactive content may be a two-dimensional code corresponding to the electronic parking ticket.

The voice password may be a voice spoken by the mobile person to the mobile device to initiate a function. For example, when the mobile person utters a voice of "find a location of the XX restaurant", the mobile device may mark the location of the XX restaurant and a navigation route from a scene map stored in advance according to the content recognized by the voice, and display them on the screen. Furthermore, when the mobile personnel sends out the voice of 'navigation leading to XX restaurant', the mobile equipment can enter a leading mode, the navigation leading mobile personnel arrives at the destination, and after the destination is reached, the voice prompt can be sent out to remind the mobile personnel, and the mobile equipment automatically returns to the initial position when the interaction instruction is not received after the preset time. The preset time may be a predetermined time period, for example, 1 minute. Optionally, when the mobile device enters the leading mode, the infrared camera may be started to perform body temperature detection on the mobile personnel, and if the mobile personnel with abnormal body temperature are detected, the current real-time position and the image of the mobile personnel with abnormal body temperature may be sent to the background to notify the background staff to perform processing.

The mobile device may further determine personalized interactive content according to the user attribute, for example, taking the user attribute as an age, when it is determined that the mobile person is a child, the interactive content may assist with interesting actions and expressions, and the like.

Optionally, in a case that the mobile device is in an idle state, a degree of deviation between the current pose of the mobile device and the initial pose may be determined; and under the condition that the deviation degree meets the preset condition, controlling the mobile equipment to adjust the current pose to the initial pose.

In order to better attract the mobile personnel to enter the activity scene, the mobile equipment is generally placed at the front door position of the activity scene, and faces away from the front door to people, but the mobile equipment tracks the target or starts a leading mode in order to acquire images in the interaction process, so that the position of the mobile equipment deviates from the preset position. Therefore, in order to increase the interaction times of the mobile equipment, the pose of the mobile equipment can be adjusted in the idle state, so that the situation that the pose adjustment in the non-idle state influences the mobile equipment to execute tasks such as interaction, guidance and the like can be avoided.

The degree of deviation may include a deflection angle and a deviation position, and the preset condition may be a condition for adjusting the pose determined in advance by a human. For example, when the deflection angle of the mobile device is greater than 30 degrees, the current pose may be adjusted to the initial pose.

Optionally, the mobile device may acquire the pose at regular intervals, so as to obtain the optimal pose information in the current application scene, and update the initial pose of the mobile device, thereby providing a better service.

According to the technical scheme, before interaction, the mobile device identifies the image corresponding to the mobile object, judges whether mobile personnel exist or not, and broadcasts according to the propaganda content corresponding to the current activity scene under the condition that the mobile personnel exist. Therefore, the interactive object of the mobile equipment can be ensured to be a person, the mobile equipment is prevented from carrying out invalid interaction with other objects, and the interactive effect is improved. In addition, the broadcasting method and the broadcasting device are used for broadcasting after the mobile object is determined to appear in the first preset area and the mobile personnel are determined to exist in the first preset area, and the first preset area is larger than the preset service area of the mobile device. Therefore, the mobile personnel outside the preset service area can be attracted to the preset service area to interact with the mobile equipment in a close range by broadcasting the propaganda content, and the interaction effect of the mobile equipment and a user can be further improved. In addition, after the mobile personnel enter the preset service area, the personalized interactive content meeting the user requirements can be determined according to the activity information corresponding to the current activity scene and/or the user attributes of the mobile personnel, so that the interactive effect of the mobile equipment and the user is further improved, and the user experience is improved. It can be seen that, in the technical scheme provided by the embodiment of the application, the interaction effect between the mobile device and the user can be improved by enriching the functions of the mobile device (determining that the interaction object is a mobile person, broadcasting the mobile person to a preset service area, and determining personalized interaction content).

In summary, as shown in fig. 2, an embodiment of the present application further provides an interaction method for a mobile device, including: S201-S207:

s201, the mobile device determines that a moving object appears in a first preset area.

S202, the mobile device acquires an image corresponding to the moving object, identifies the image corresponding to the moving object and judges whether a moving person exists or not.

And S203, the mobile device determines that mobile personnel exist, and broadcasts according to the propaganda content corresponding to the current activity scene.

S204, under the condition that the mobile equipment is in a non-peak period in the current time period, determining that at least two mobile personnel enter a second preset area.

S205, the mobile device acquires target images of at least two mobile personnel and extracts group characteristic information of the at least two mobile personnel in the target images.

S206, the mobile device determines that at least two mobile persons belong to the same group according to the group characteristic information, and determines the group types of the at least two mobile persons.

S207, under the condition that the mobile device determines that the mobile personnel enter the preset service area, the mobile device determines interactive contents corresponding to the group categories to which at least two mobile personnel belong from the interactive content library, and controls the mobile device to interact with the mobile personnel according to the interactive contents.

Optionally, as shown in fig. 3, an embodiment of the present application further provides an interaction method for a mobile device, including: S301-S307:

s301, the mobile device determines that a moving object appears in a first preset area.

S302, the mobile device controls the body of the mobile device or the acquisition sub-device of the mobile device to rotate according to the position of the mobile object, and images corresponding to the mobile object are acquired.

And S303, the mobile device identifies the image corresponding to the moving object, and broadcasts the image according to the propaganda content corresponding to the current activity scene when the existence of the moving personnel is determined.

S304, under the condition that the mobile device determines that the mobile personnel enters the second preset area, target tracking is carried out on the mobile personnel according to the continuous frame images of the mobile personnel.

S305, the mobile device determines whether the mobile personnel enter a third preset area or not according to the tracking data acquired in the target tracking process, and performs feature recognition on the continuous frame images under the condition that the mobile personnel enter the third preset area, so as to determine the user attribute of the mobile personnel.

S306, the mobile device determines that the mobile personnel enters a preset service area according to the tracking data acquired in the target tracking process.

S307, the mobile device determines interactive content according to the activity information corresponding to the current activity scene and/or the user attribute of the mobile personnel, and controls the mobile device to interact with the mobile personnel according to the interactive content.

As shown in fig. 4, an embodiment of the present application further provides a mobile device, where the mobile device may include: the device comprises a detection module 11, an acquisition module 12, a broadcast module 13 and an interaction module 14.

The detection module 11 executes S101 in the above method embodiment, the acquisition module 12 executes S102 in the above method embodiment, the broadcast module 13 executes S103 in the above method embodiment, and the interaction module 14 executes S104 in the above method embodiment.

Specifically, the detecting module 11 is configured to detect and determine a moving object in a first preset area, where the first preset area is larger than a preset service area; the acquisition module 12 is used for acquiring an image corresponding to the moving object, identifying the image corresponding to the moving object through the identification module, and judging whether a moving person exists or not; the broadcasting module 13 is used for broadcasting according to the propaganda content corresponding to the current activity scene under the condition that the identification module determines that the mobile personnel exist; and the interaction module 14 is configured to determine interaction content according to the activity information corresponding to the current activity scene and/or the user attribute of the mobile person when it is determined that the mobile person enters the preset service area, and control the mobile device to interact with the mobile person according to the interaction content. The interactive module 14 may include a display screen and/or a player, and the interactive module 14 may be integrated with the broadcast module 13, or may be separately disposed.

Optionally, in a possible implementation manner, the user attribute includes a group category, and the mobile device provided in the present application further includes a determining module and a determining module;

the determining module is used for determining that at least two mobile personnel enter a second preset area under the condition that the current time period is determined to be in the non-peak period, and the second preset area is smaller than the first preset area and larger than a preset service area;

the acquisition module 12 is further configured to acquire target images of at least two mobile people and extract group feature information of the at least two mobile people in the target images;

the judging module is used for judging whether at least two mobile personnel belong to the same group according to the group characteristic information;

and the determining module is used for determining the group category of the at least two mobile personnel under the condition that the judging module judges that the at least two mobile personnel belong to the same group.

Optionally, in another possible implementation manner, the determining module is specifically configured to: and determining interactive contents corresponding to group categories to which at least two mobile personnel belong from an interactive content library, wherein the interactive content library comprises the corresponding relation between the group categories and the interactive contents.

Optionally, in another possible implementation manner, the mobile device provided by the present application further includes a tracking module;

the tracking module is used for tracking the target of the mobile personnel according to the continuous frame images of the mobile personnel under the condition that the determining module determines that the mobile personnel enters a second preset area, and the second preset area is smaller than the first preset area and larger than a preset service area;

the determining module is further used for determining whether the mobile personnel enters a preset service area according to the tracking data acquired in the target tracking process.

Alternatively, in another possible implementation,

the determining module is further used for determining whether the mobile personnel enters a third preset area according to the tracking data acquired in the target tracking process, wherein the third preset area is smaller than the second preset area and larger than a preset service area;

the determining module is further used for carrying out feature recognition on the continuous frame images under the condition that the mobile personnel is determined to enter the third preset area, and determining the user attribute of the mobile personnel, wherein the user attribute comprises at least one of sex, age, height, historical consumption information and expected consumption level.

Optionally, in another possible implementation manner, the tracking module is specifically configured to: acquiring continuous frame images of the moving personnel according to a preset frequency; and adjusting acquisition parameters according to the difference of at least two frames of images in the continuous frames of images, and tracking the target of the moving person.

Optionally, in another possible implementation manner, the determining module is specifically configured to: detecting a moving object in a detection area of a mobile device; if the moving objects in the plurality of sub-areas in the detection area are detected, determining the sub-area meeting the preset angle condition in the plurality of sub-areas as a first preset area.

Optionally, in another possible implementation manner, the obtaining module 12 is specifically configured to: and controlling the body of the mobile equipment or the acquisition sub-equipment of the mobile equipment to rotate according to the position of the mobile object, and acquiring an image corresponding to the mobile object.

Optionally, in another possible implementation manner, the mobile device provided by the present application further includes a control module;

the determining module is further used for determining the deviation degree of the current pose of the mobile equipment and the initial pose under the condition that the mobile equipment is in an idle state;

and the control module is used for controlling the mobile equipment to adjust the current pose to the initial pose under the condition that the deviation degree determined by the determination module meets the preset condition.

Optionally, in another possible implementation manner, the determining module is specifically configured to:

responding to a trigger instruction, determining interactive content according to the activity information, and triggering the trigger instruction according to the trigger operation of the mobile personnel or a voice password;

or, in response to a trigger instruction, determining interactive content according to the user attribute;

or responding to the trigger instruction, screening target activity information from the activity information according to the user attribute, and determining interactive content according to the target activity information.

Optionally, the mobile device may further include a storage module, which is used for storing program codes of the mobile device, and the like.

As shown in fig. 5, the embodiment of the present application further provides an interaction apparatus for a mobile device, which includes a memory 41, a processor 42 (42-1 and 42-2), a bus 43, and a communication interface 44; the memory 41 is used for storing computer execution instructions, and the processor 42 is connected with the memory 41 through a bus 43; when the interaction means of the mobile device is running, the processor 42 executes the computer executable instructions stored in the memory 41, so that the interaction means of the mobile device performs the interaction method of the mobile device as provided in the above embodiments.

In particular implementations, processor 42 may include one or more Central Processing Units (CPUs), such as CPU0 and CPU1 shown in FIG. 5, as an example. And as an example, the interaction means of the mobile device may comprise a plurality of processors 42, such as processor 42-1 and processor 42-2 shown in fig. 5. Each of the processors 42 may be a single-Core Processor (CPU) or a multi-Core Processor (CPU). Processor 42 may refer herein to one or more devices, circuits, and/or processing cores that process data (e.g., computer program instructions).

The memory 41 may be, but is not limited to, a read-only memory 41 (ROM) or other type of static storage device that can store static information and instructions, a Random Access Memory (RAM) or other type of dynamic storage device that can store information and instructions, an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compact disc, laser disc, optical disc, digital versatile disc, blu-ray disc, etc.), magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. The memory 41 may be self-contained and coupled to the processor 42 via a bus 43. The memory 41 may also be integrated with the processor 42.

In a specific implementation, the memory 41 is used for storing data in the present application and computer-executable instructions corresponding to software programs for executing the present application. The processor 42 may operate or execute software programs stored in the memory 41 and invoke data stored in the memory 41 to perform various functions of the interaction means of the mobile device.

The communication interface 44 is any device, such as a transceiver, for communicating with other devices or communication networks, such as a control system, a Radio Access Network (RAN), a Wireless Local Area Network (WLAN), and the like. The communication interface 44 may include a receiving unit implementing a receiving function and a transmitting unit implementing a transmitting function.

The bus 43 may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an extended ISA (enhanced industry standard architecture) bus, or the like. The bus 43 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is shown in FIG. 5, but this is not intended to represent only one bus or type of bus.

As an example, in conjunction with fig. 4, the acquiring module in the mobile device implements the same functions as the receiving unit in fig. 5, the interacting module in the mobile device implements the same functions as the processor in fig. 5, and the storage module in the mobile device implements the same functions as the memory in fig. 5.

For the explanation of the related contents in this embodiment, reference may be made to the above method embodiments, which are not described herein again.

Through the description of the foregoing embodiments, it will be clear to those skilled in the art that, for convenience and simplicity of description, only the division of the functional modules is illustrated, and in practical applications, the above function distribution may be completed by different functional modules as needed, that is, the internal structure of the apparatus may be divided into different functional modules to complete all or part of the above described functions. For the specific working processes of the system, the apparatus and the unit described above, reference may be made to the corresponding processes in the foregoing method embodiments, and details are not described here again.

The embodiment of the present application further provides a computer-readable storage medium, where instructions are stored in the computer-readable storage medium, and when the instructions are executed by a computer, the computer is enabled to execute the interaction method of the mobile device provided in the foregoing embodiment.

The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of the computer readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM), a register, a hard disk, an optical fiber, a CD-ROM, an optical storage device, a magnetic storage device, any suitable combination of the foregoing, or any other form of computer readable storage medium known in the art. An exemplary storage medium is coupled to the processor such the processor can read information from, and write information to, the storage medium. Of course, the storage medium may also be integral to the processor. The processor and the storage medium may reside in an Application Specific Integrated Circuit (ASIC). In embodiments of the present application, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.

The above description is only an embodiment of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An interaction method of a mobile device, comprising:

determining that a mobile object appears in a first preset area, wherein the first preset area is larger than a preset service area;

acquiring an image corresponding to the moving object, identifying the image corresponding to the moving object, and judging whether a moving person exists or not;

if yes, broadcasting according to the propaganda content corresponding to the current activity scene;

under the condition that the current time period is determined to be in a non-peak period, determining that at least two mobile personnel enter a second preset area, wherein the second preset area is smaller than the first preset area and larger than the preset service area;

acquiring target images of the at least two mobile personnel, and extracting group characteristic information of the at least two mobile personnel in the target images;

judging whether the at least two mobile personnel belong to the same group according to the group characteristic information;

if the mobile personnel belong to the group, determining the group category to which the at least two mobile personnel belong;

determining interactive content according to activity information corresponding to the current activity scene and user attributes of the at least two mobile personnel or determining the interactive content according to the user attributes under the condition that the at least two mobile personnel are determined to enter the preset service area; then controlling the mobile equipment to interact with the at least two mobile personnel according to the interactive content; the user attributes include a group category.

2. The interaction method of mobile device of claim 1, wherein determining the interaction content according to the user attributes of the at least two mobile people comprises:

and determining interactive contents corresponding to the group categories to which the at least two mobile personnel belong from an interactive content library, wherein the interactive content library comprises the corresponding relation between the group categories and the interactive contents.

3. The interaction method of the mobile device according to claim 1, wherein after the broadcasting according to the promotion content corresponding to the current activity scene, the method further comprises:

under the condition that the mobile personnel enter a second preset area, carrying out target tracking on the mobile personnel according to continuous frame images of the mobile personnel, wherein the second preset area is smaller than the first preset area and larger than the preset service area;

and determining whether the mobile personnel enters the preset service area or not according to the tracking data acquired in the target tracking process.

4. The method of claim 3, wherein before determining whether the mobile person enters the predetermined service area, the method further comprises:

determining whether the mobile personnel enters a third preset area according to the tracking data acquired in the target tracking process, wherein the third preset area is smaller than the second preset area and larger than the preset service area;

and under the condition that the mobile person is determined to enter the third preset area, performing feature recognition on the continuous frame images, and determining user attributes of the mobile person, wherein the user attributes comprise at least one of sex, age, height, historical consumption information and expected consumption level.

5. The interactive method of mobile device of claim 3, wherein said tracking the object of the mobile person according to the continuous frame images of the mobile person comprises:

acquiring continuous frame images of the mobile personnel according to a preset frequency;

and adjusting acquisition parameters according to the difference of at least two images in the continuous frame images, and tracking the target of the moving person.

6. The interaction method of mobile device of claim 1, wherein said determining that a moving object is present in a first predetermined area comprises:

detecting a moving object in a detection area of the mobile device;

if the moving objects are detected to appear in the plurality of sub-areas in the detection area, determining the sub-area meeting the preset angle condition in the plurality of sub-areas as the first preset area.

7. The method of claim 1, wherein the obtaining of the image corresponding to the moving object comprises:

and controlling the body of the mobile equipment or the acquisition sub-equipment of the mobile equipment to rotate according to the position of the mobile object, and acquiring the image corresponding to the mobile object.

8. The method of claim 1, further comprising:

determining a degree of deviation of a current pose of the mobile device from an initial pose when the mobile device is in an idle state;

and under the condition that the deviation degree meets a preset condition, controlling the mobile equipment to adjust the current pose to the initial pose.

9. The interaction method of the mobile device according to any one of claims 1 to 8, wherein the determining the interaction content according to the activity information corresponding to the current activity scenario and the user attributes of the at least two mobile people, or determining the interaction content according to the user attributes comprises:

responding to a trigger instruction, and determining the interactive content according to the user attribute; the trigger instruction is triggered according to the trigger operation or the voice password of the mobile personnel;

alternatively, the first and second electrodes may be,

and responding to the trigger instruction, screening target activity information from the activity information according to the user attribute, and determining the interactive content according to the target activity information.

10. A mobile device, comprising:

the mobile terminal comprises a detection module, a processing module and a processing module, wherein the detection module is used for detecting and determining a mobile object in a first preset area, and the first preset area is larger than a preset service area;

the acquisition module is used for acquiring an image corresponding to the moving object, identifying the image corresponding to the moving object through the identification module and judging whether a moving person exists or not;

the broadcasting module is used for broadcasting according to the propaganda content corresponding to the current activity scene under the condition that the identification module determines that the mobile personnel exist;

the determining module is used for determining that at least two mobile personnel enter a second preset area under the condition that the current time interval is determined to be in a non-peak period, and the second preset area is smaller than the first preset area and larger than the preset service area;

the acquisition module is further configured to acquire target images of the at least two mobile people and extract group feature information of the at least two mobile people in the target images;

the judging module is used for judging whether the at least two mobile personnel belong to the same group according to the group characteristic information;

the determining module is further configured to determine a group category to which the at least two mobile people belong when the determining module determines that the at least two mobile people belong to the same group;

the interaction module is used for determining interaction content according to the activity information corresponding to the current activity scene and the user attributes of the at least two mobile personnel or determining the interaction content according to the user attributes under the condition that the at least two mobile personnel are determined to enter the preset service area; then controlling the mobile equipment to interact with the at least two mobile personnel according to the interactive content; the user attributes include a group category.

11. An interaction device of mobile equipment is characterized by comprising a memory, a processor, a bus and a communication interface; the memory is used for storing computer execution instructions, and the processor is connected with the memory through the bus;

when the interaction device of the mobile device is operated, the processor executes the computer-executable instructions stored in the memory to cause the interaction device of the mobile device to perform the interaction method of the mobile device according to any one of claims 1 to 9.

12. A computer-readable storage medium having stored therein instructions, which when executed by a computer, cause the computer to perform the method of interaction of a mobile device according to any one of claims 1-9.