CN112308869A

CN112308869A - Image acquisition method and device, electronic equipment and computer storage medium

Info

Publication number: CN112308869A
Application number: CN201910697213.7A
Authority: CN
Inventors: 毛璐娜; 宫晨; 周立; 周士天
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2019-07-30
Filing date: 2019-07-30
Publication date: 2021-02-02
Also published as: WO2021018019A1

Abstract

The embodiment of the invention provides a shelf image acquisition method, which comprises the following steps: acquiring a shelf image acquired according to the indication of first guide information, wherein the shelf is used for bearing commodities, and the first guide information is used for indicating an image acquisition path of the shelf; obtaining an edge detection result of the shelf edge detection of the shelf image; and if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition. By the embodiment of the invention, the image acquisition quality of the goods shelf can be improved.

Description

Image acquisition method and device, electronic equipment and computer storage medium

Technical Field

The embodiment of the invention relates to the technical field of computers, in particular to an image acquisition method, an image acquisition device, electronic equipment and a computer storage medium.

Background

In the prior art, in some image acquisition scenes, because the length and/or height of an acquired target object is large, a complete image of the target object cannot be acquired through one image acquisition operation, so that a user needs to perform multiple image acquisition operations to acquire images of different parts of the target object, and therefore comprehensive image acquisition of the target object is achieved.

For example, when the shelf display information of the next store is digitized, it is necessary to take a complete shelf image, recognize the shelf image by the artificial intelligence technology, and generate the digitized display information according to the recognition result. When a shelf image is shot, the shelf space is narrow, and the length of the whole shelf is long, so that the whole shelf is difficult to be shot completely through one picture and clear goods information is obtained, and the shelf needs to be shot for many times in order to solve the problem.

In the process of multiple shooting, due to reasons such as irregular user operation, partial areas of the target object are omitted during image acquisition, and the complete image and information of the target object cannot be obtained.

Disclosure of Invention

In view of the above, embodiments of the present invention provide an image capturing scheme to solve some or all of the above problems.

According to a first aspect of the embodiments of the present invention, there is provided a shelf image capturing method, including: acquiring a shelf image acquired according to the indication of first guide information, wherein the shelf is used for bearing commodities, and the first guide information is used for indicating an image acquisition path of the shelf; obtaining an edge detection result of the shelf edge detection of the shelf image; and if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition.

According to a second aspect of the embodiments of the present invention, there is provided a commodity information processing method including: acquiring image data of a shelf according to acquired first guide information, wherein the first guide information is used for indicating an image acquisition path of the shelf; identifying the image data, and acquiring commodity information on the shelf and information whether the commodity information contains the edge of the shelf; and if the image data contains the information of the edge of the shelf, judging whether all the acquired image data contain all the commodity information according to the commodity information, and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition according to a judgment result.

According to a third aspect of the embodiments of the present invention, there is provided a shelf image capturing method including: displaying first acquisition prompt information of the goods on the shelf, wherein the first acquisition prompt information is used for indicating an acquisition position when the goods on the shelf are subjected to image acquisition along an image acquisition path; acquiring an image for image acquisition according to the first acquisition prompt information, and identifying the acquired image; and if the identification result indicates that the image comprises the shelf edge, displaying second acquisition prompt information for indicating a new image acquisition path and indicating to continue image acquisition.

According to a fourth aspect of the embodiments of the present invention, there is provided a client, including: the display interface is used for displaying first acquisition prompt information, and the first acquisition prompt information is used for indicating image acquisition of the target object along an image acquisition path; the display interface is further configured to display second acquisition prompt information, where the second acquisition prompt information indicates that image acquisition is performed on the target object along a new image acquisition path when the acquired image includes an edge of the target object.

According to a fifth aspect of the embodiments of the present invention, there is provided a commodity information processing method including: collecting image data of a shelf; processing the image data, and identifying to obtain commodity information on the shelf; and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

According to a sixth aspect of the embodiments of the present invention, there is provided a method for processing commodity information, including: calling an image acquisition device of a client to shoot image data of a shelf in response to shooting operation initiated by a user; processing the image data, and identifying to obtain commodity information on the shelf; and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

According to a seventh aspect of the embodiments of the present invention, there is provided a processing method of commodity replenishment, including: calling an image acquisition device to shoot image data of the goods shelf in response to a replenishment operation initiated by a user; carrying out identification processing on the image data to obtain commodity information on the goods shelf; and determining the goods to be replenished according to the goods information on the goods shelf.

According to an eighth aspect of the embodiments of the present invention, there is provided an image capturing method, including: obtaining a detection result of real-time target object edge detection on an acquired image, wherein the acquired image comprises partial image information of a target object; if the detection result indicates that the edge of the target object is detected in the image, acquiring attitude data of image acquisition equipment for acquiring the image; and generating corresponding guide information according to the attitude data, and guiding a user to perform continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using the acquired multiple images.

According to a ninth aspect of the embodiments of the present invention, there is provided an image capturing method including: acquiring attitude data of image acquisition equipment in the process of acquiring an image of a target object; and generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information.

According to a tenth aspect of the embodiments of the present invention, there is provided an image pickup apparatus including: the detection module is used for acquiring a detection result of real-time target object edge detection on an acquired image, wherein the acquired image comprises partial image information of a target object; a first obtaining module, configured to obtain pose data of an image acquisition device that acquires the image if the detection result indicates that the edge of the target object is detected in the image; and the generating module is used for generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using a plurality of acquired images.

According to an eleventh aspect of embodiments of the present invention, there is provided an electronic apparatus including: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus; the memory is used for storing at least one executable instruction, and the executable instruction causes the processor to execute the corresponding operation of the method according to any one of the first aspect to the third aspect and the fifth aspect to the ninth aspect.

According to a twelfth aspect of embodiments of the present invention, there is provided a computer storage medium having stored thereon a computer program which, when executed by a processor, implements the method as described in any one of the first to third aspects and the fifth to ninth aspects.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, it is obvious that the drawings in the following description are only some embodiments described in the embodiments of the present invention, and it is also possible for a person skilled in the art to obtain other drawings based on the drawings.

Fig. 1 is a flowchart illustrating steps of an image capturing method according to a first embodiment of the present invention;

FIG. 2 is a flowchart illustrating steps of an image capturing method according to a second embodiment of the present invention;

FIG. 3 is a flowchart illustrating steps of an image capturing method according to a third embodiment of the present invention;

FIG. 4 is a flowchart illustrating steps of an image capturing method according to a fourth embodiment of the present invention;

FIG. 5a is a flowchart of the steps of scenario one of the present invention;

FIG. 5b is a flowchart illustrating steps of a second exemplary embodiment of the present invention;

FIG. 5c is a diagram illustrating a segmentation path using scenario two in accordance with the present invention;

FIG. 5d is a schematic view of a capture interface using scenario two of the present invention;

FIG. 5e is a flowchart illustrating a third step of a usage scenario of the present invention;

FIG. 5f is a flowchart illustrating steps in scenario four of the present invention;

FIG. 5g is a schematic diagram of a display interface of a client using scenario five of the present invention;

FIG. 5h is a flowchart of steps for use scenario six of the present invention;

FIG. 5i is a flowchart illustrating steps in a seventh aspect of the present invention;

FIG. 5j is a flowchart illustrating steps in a usage scenario eight of the present invention;

FIG. 5k is an information interaction diagram of a user, an image capture device, and a server according to scenario nine of the present invention;

FIG. 6 is a flowchart illustrating steps of an image capturing method according to a fifth embodiment of the present invention;

FIG. 7 is a flowchart illustrating steps of an image capturing method according to a sixth embodiment of the present invention;

fig. 8 is a block diagram of an image capturing device according to a seventh embodiment of the present invention;

fig. 9 is a block diagram of an image capturing apparatus according to an eighth embodiment of the present invention;

fig. 10 is a schematic structural diagram of an electronic device according to a ninth embodiment of the present invention.

Detailed Description

In order to make those skilled in the art better understand the technical solutions in the embodiments of the present invention, the technical solutions in the embodiments of the present invention will be described clearly and completely with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all embodiments. All other embodiments obtained by a person skilled in the art based on the embodiments of the present invention shall fall within the scope of the protection of the embodiments of the present invention.

The following further describes specific implementation of the embodiments of the present invention with reference to the drawings.

Example one

Referring to fig. 1, a flowchart illustrating steps of an image capturing method according to a first embodiment of the present invention is shown.

The image acquisition method of the embodiment comprises the following steps:

step S102: and obtaining a detection result of real-time target object edge detection on the acquired image.

The acquired image comprises partial image information of the target object, and the target object edge detection is used for detecting whether the acquired image comprises the edge of the target object.

The detection of the target object edge can be carried out at the client, and the detection result can also be sent to the client after the detection is carried out at the server. This may be implemented using any suitable model or algorithm or other means, for example, using a trained Neural Network model capable of performing edge detection on the target object, such as a Convolutional Neural Network (CNN), to perform edge detection on the acquired image.

For another example, feature extraction is performed on the acquired image by using a feature extraction algorithm, and whether the edge of the target object is included in the image is determined according to the extracted features, so as to generate a detection result.

Whether the user collects the image of the edge of the target object can be judged according to the detection result, and then reference is provided for subsequently generating proper guide information so as to guide the user, so that the phenomenon that the user performs wrong operation when collecting the image is avoided, and the fact that complete image information can be collected is ensured.

For example, the detection result indicates that the edge of the target object is detected in the image, step S104 is performed; otherwise, guidance information for instructing the user to continue moving in the current moving direction and photographing may be directly generated.

Step S104: if the detection result indicates that the edge of the target object is detected in the image, acquiring attitude data of an image acquisition device acquiring the image.

The pose data of the image capturing device, including but not limited to acceleration information and/or angular velocity information in a spatial coordinate system, is used to characterize the state in which it is currently located. For example, it is determined from the acceleration information and/or the angular velocity information that the image capturing device is currently in a 45 degree tilt-up state, and so on.

And determining whether the user has the intention of continuously shooting different positions of the target object according to the attitude data, wherein the generated guide information is matched with the intention so as to guide the user to accurately acquire the image and ensure that the complete image information of the target object can be acquired.

Step S106: and generating corresponding guide information according to the attitude data, and guiding a user to perform continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using the acquired multiple images.

In a specific implementation manner, a corresponding relationship between the posture data and the guidance information or the guidance keyword may be set in the image capturing device, and accordingly, the corresponding guidance information may be determined and generated according to the posture data, or the corresponding guidance keyword may be determined according to the posture data, and then, the corresponding guidance information may be generated according to the guidance keyword. For example, if the pose data corresponds to the guide keyword "move up", guide information such as "please move up one frame to shoot" may be generated. Through the guide information, the user can be effectively guided to carry out continuous image acquisition, so that complete image information of the target object can be formed according to a plurality of acquired images in the following process.

It should be noted that, in the process of acquiring the complete image information, steps S102 to S106 may be repeatedly executed for a plurality of times until it is determined that the user does not have an intention to continue the acquisition according to the posture data of the image acquisition apparatus acquired in step S104, and guidance information instructing the user to end the image acquisition may be generated. After the acquisition is finished, the acquired plurality of images may be used to form complete image information of the target object.

It should be noted that, for the case that the target object is a shelf, the image acquisition method in this embodiment is particularly suitable for use scenes where goods on the shelf are placed irregularly. For example, the goods shelves in a small retail store have the problems of tight shelf placement and messy and irregular goods, the image acquisition method of the embodiment can overcome the problems, realize the acquisition of the complete image of the goods shelves, and can obtain clear goods information so as to determine the goods on the goods shelves through recognizing the complete image in the follow-up process.

The unmanned aerial vehicle can be adopted for a large market scene or a large-scale shooting scene, but the technical means of unmanned aerial vehicle shooting cannot be applied to the use scene of the application, and the technical means cannot be transferred to the use scene of the application. Similarly, the technical means in the electronic price tag scenario is also difficult to implement in the present usage scenario.

According to the embodiment, the edge of the target object is detected in real time on the acquired image, when the acquired image contains the edge of the target object, the attitude data of the image acquisition equipment is acquired, and then the corresponding guide information is generated according to the attitude data, so that the user is guided to acquire the image in a standard manner through the guide information, the purpose of finally completing the image acquisition of a plurality of parts contained in the whole target object is achieved, omission is avoided, and the purpose of acquiring the complete image of the target object is achieved.

The image acquisition method of the present embodiment may be performed by any suitable electronic device with data processing capabilities, including but not limited to: servers, mobile terminals (such as tablet computers, mobile phones and the like), PCs and the like.

Example two

Referring to fig. 2, a flowchart illustrating steps of an image capturing method according to a second embodiment of the present invention is shown.

The image capturing method of the present embodiment includes the aforementioned steps S102 to S106.

Wherein, before step S102, the method further comprises:

step S100: and acquiring a lightweight neural network model which is dynamically issued to the image acquisition equipment and is used for carrying out the edge detection of the target object.

It should be noted that this step is an optional step. If this step is performed, it may be performed at any suitable timing prior to step S102.

In this embodiment, in order to ensure that whether the acquired image includes the edge of the target object can be detected in time to ensure the accuracy of the generated guidance information, the image acquisition device locally has a dynamically issued lightweight neural network model, and the image can be detected locally by using the lightweight neural network model without being transmitted to the server, so that the detection speed and efficiency are greatly improved.

The lightweight neural network model is also called a miniature neural network model, and refers to a neural network model with less required parameters and less calculation cost. The method has low calculation cost, so the method can be deployed on image acquisition equipment with limited calculation resources.

Specifically, the lightweight neural network model may be a lightweight convolutional neural network model trained in advance. The convolutional neural network model has an input layer, a hidden layer, and an output layer. When the convolutional neural network model is trained, a large number of images which contain target objects (such as shelves, large mechanical equipment, large containers and the like) and are collected in advance are used for labeling the images, the edges of the target objects are mainly labeled, and then the convolutional neural network model is trained by using the labeled images. And the trained convolutional neural network model can correctly identify whether the image contains the edge of the shelf or not. And then dynamically issuing the trained convolutional neural network model to image acquisition equipment.

By adopting the form of the lightweight neural network model, on the premise of ensuring that the edge of the target object can be accurately detected on the image, the requirement on the computing capacity and the requirement on the storage space are reduced, so that the scheme can adapt to more image acquisition devices, especially small-sized image acquisition devices such as mobile phones, tablet computers and other mobile terminal devices.

In this embodiment, in the case of executing step S100, the step S102 may be implemented as: and carrying out real-time target object edge detection on the acquired image by using the lightweight neural network model to obtain a detection result. And indicating whether the edge of the target object is contained in the current image or not according to the detection result.

Therefore, the edge detection of the target object can be quickly, efficiently and accurately carried out locally on the image acquisition equipment, and the timeliness of generating the guide information is further ensured.

In addition, through the image acquisition equipment that sends down the light-weight neural network model developments, make it carry out real-time target object edge detection to the image of gathering locally, under the prerequisite of guaranteeing detectability, the promptness of detection has been promoted, and then the promptness of follow-up guide information generation has been guaranteed, compare in the mode that needs to send the image of gathering to the backstage server side in the past and detect and return the testing result, detect locally at image acquisition equipment, need not to receive the restriction of network transmission speed, the reliability is better, speed and efficiency are also higher.

EXAMPLE III

Referring to fig. 3, a flowchart illustrating steps of an image capturing method according to a third embodiment of the present invention is shown.

The method may or may not include step S100, and when step S100 is included, step S102 may be implemented by using the implementation manner in embodiment two.

In this embodiment, the step S104 of acquiring the pose data of the image capturing device for capturing the image may be implemented as: and acquiring acceleration information and/or angular velocity information of the image acquisition equipment in a space coordinate system.

For example, the spatial coordinate system includes an X-axis, a Y-axis, and a Z-axis, and acceleration information on the three axes can be acquired by an accelerometer provided in the image capturing apparatus. The angular velocity information on these three axes may be acquired by a gyroscope provided in the image pickup device.

Of course, for image acquisition devices with different structures, acceleration information and/or angular velocity information in a spatial coordinate system may be acquired in different manners, which is not limited in this embodiment.

Whether the image acquisition equipment inclines upwards or downwards can be determined according to the acceleration information and/or the angular velocity information of the image acquisition equipment in the space coordinate system, and further whether a user has the intention of line feed shooting can be determined.

In one case, if the image capturing device maintains a certain inclination angle or rapidly tends to retract the image capturing device within a period of time after capturing the edge of the target object, it indicates that the user has captured the complete image information of the target object without the intention of line feed shooting, and may generate guidance information for guiding the user to end image capturing.

In another case, if the image capturing apparatus has an upward or downward inclination angle change within a period of time after capturing the edge of the target object, it indicates that the user has an intention to perform line feed photographing, and thus step S106 may be performed to generate guidance information for guiding the user to perform a subsequent image capturing.

Optionally, when the foregoing implementation manner is adopted in the step S104, the step S106 includes the following sub-steps:

substep S1061: and determining the current posture of the image acquisition equipment according to the acceleration information and/or the angular velocity information.

Whether the image acquisition device moves and/or rotates can be determined according to the acceleration information of the image acquisition device on the X axis, the Y axis and the Z axis. From its angular velocity information in the X, Y and Z axes, its inclination from horizontal or vertical can be determined. The current attitude of the image acquisition device can be determined from the acceleration information and/or the angular velocity information.

For example, the gyroscope has the X-axis pointing horizontally to the right, the Y-axis pointing vertically to the front, and the Z-axis pointing directly above the screen of the image capture device.

When it is determined that the image capturing device has an angular velocity on the X axis from the angular velocity information, it may be determined that it is in an inclined state, and the current posture may be determined as being inclined upward or downward from the value of the angular velocity. Similarly, when it is determined that there is acceleration in the Z axis from the acceleration information, it can be determined that the image pickup device has a tendency to move upward.

Substep S1062: and generating guide information for indicating the user to move to the direction matched with the current posture so as to collect continuous images according to the current posture.

The image acquisition equipment can be provided with a corresponding relation between the current posture and the guide information or the guide keywords, and after the current posture is determined, the matched guide information or the matched guide keywords can be determined according to the current posture and the set corresponding relation, so that the guide information which is moved in the direction matched with the current posture to perform continuous image acquisition is generated.

For example, if the current posture is tilted upward and the matching guidance keyword is "moved upward", guidance information indicating that the user moves upward and continues image capturing may be generated.

For another example, if the current posture is downward tilting and the matched guidance keyword is "downward shifting", guidance information indicating that the user moves downward and continues image capturing may be generated.

Of course, it is also possible to directly match to guidance information such as "please move up to shoot" or "please move down to shoot".

Therefore, accurate guide information can be generated to indicate a user to continuously shoot through the guide information, and therefore complete image information of the target object can be acquired.

According to the embodiment, the edge detection of the target object is carried out on the acquired image in real time, when the acquired image contains the edge of the target object, the attitude data of the image acquisition equipment is acquired, and then the corresponding guide information is generated according to the attitude data, so that the user is guided to carry out the image acquisition in a standard manner through the guide information, the purpose of finally completing the image acquisition of a plurality of parts contained in the whole target object is achieved, omission is avoided, and the purpose of obtaining the complete image of the target object is achieved

Example four

Referring to fig. 4, a flowchart illustrating steps of an image capturing method according to a fourth embodiment of the present invention is shown.

The method may or may not include step S100, and when step S100 is included, step S102 may be implemented by using the implementation manner in embodiment two. Step S104 may adopt the implementation manner of the third embodiment or other implementation manners. When the implementation manner of the third embodiment is adopted in step S104, the implementation manner of the third embodiment may be adopted in step S106.

In the present embodiment, if it is determined from the posture data of the image pickup device acquired in step S104 that the user does not have an intention to photograph other parts of the target object, the entire image information of the target object may be formed using the plurality of images picked up.

In one possible approach, forming complete image information of the target object using the acquired plurality of images includes: and splicing the plurality of acquired images to obtain a complete image containing complete image information of the target object.

Because the mode of collecting in batches is adopted in the image collecting process of the target object, a complete image containing complete image information of the target object can be obtained by splicing a plurality of collected images, and a user can observe the target object more intuitively due to the complete image at the spliced part.

Specifically, the stitching the plurality of acquired images to obtain a complete image including complete image information of the target object includes the following steps:

step S108: from the plurality of acquired images, a plurality of sets of images having an image registration relationship are determined.

In the present embodiment, two images are included in each set of images. The overlapping relation between the images indicates that the two images are adjacent in space position, so that the relative position relation between the images can be deduced according to the overlapping relation, and the images are spliced according to the relative position relation. By adopting the splicing mode, the accuracy of splicing can be ensured, and quick splicing can be realized.

One way to determine the sets of images having an image registration relationship is to: extracting the characteristics of each image in the plurality of collected images to obtain characteristic points corresponding to each image; and matching any two images according to the characteristic points of the two images, and determining the multiple groups of images with the image coincidence relation based on the matching result.

The feature extraction of the image may be performed by any suitable algorithm such as a HOG (Histogram of Oriented Gradient) feature extraction algorithm, an LBP (Local Binary Pattern) feature extraction algorithm, or a Haar-like feature extraction algorithm.

When any two images are matched according to the feature points of the two images, the matching result can be determined by comparing whether the similarity of the two images meets a certain threshold (specifically, for example, calculating the distance between the feature points to determine the similarity). If the distance between the characteristic points of the two images is smaller than a preset value, the matching result indicates that the two images have a coincidence relation; on the contrary, if the distance between the characteristic points of the two images is larger than or equal to the set value, the matching result indicates that the two images do not have the superposition relationship.

By adopting the method, the images with the coincidence relation can be accurately judged, so that the accuracy of splicing is ensured.

Step S110: and splicing the plurality of acquired images according to the image coincidence relation, and acquiring a complete image containing complete image information of the target object according to a splicing result.

In a specific implementation, adjacent images of each image are determined according to the coincidence relation between the images, and the relative position relation is determined according to the position of the coincident part in the two adjacent images, so that the multiple images are spliced, and a complete image is obtained. The complete image contains the complete image information of the target object.

If it is determined that the image a and the image B have overlapping portions according to the overlapping relationship, and the overlapping portions are located on the left side of the image a and the right side of the image B, the image a can be stitched on the right side of the image B.

For another example, if there is an overlap between the upper side of the image a and the lower side of the image C, the image C is stitched to the upper side of the image a.

Subsequent users can analyze and/or process the complete image as needed to obtain the needed information. For example, the target object is a shelf, and it can be determined whether a certain type of product is placed in a position convenient for taking by analyzing a complete image of the shelf, and the like.

It should be noted that the image stitching may also be completed by the server, for example, the image acquisition device uploads a plurality of acquired images to the background server, the server performs corresponding identification and stitching operations, and then sends the completed images back to the image acquisition device, so as to reduce the data processing burden of the image acquisition device.

In addition, the lightweight neural network model is dynamically issued to the image acquisition equipment, so that the acquired image can be subjected to real-time target object edge detection locally, on the premise of ensuring the detection performance, the detection timeliness is improved, and further the timeliness and the accuracy of subsequent guide information generation are ensured.

In addition, the collected multiple images are spliced to form a complete image containing complete image information, so that a user can observe a target object more directly, and the complete image can be analyzed and processed as required to obtain a required analysis result.

Using scenario one:

referring to fig. 5a, a flow chart of the steps of the image acquisition method in the first usage scenario is shown.

In this usage scenario, the image capturing method will be described by taking the image capturing device as a mobile phone and the target object as a shelf as an example. Specifically, the image acquisition method comprises the following steps:

step A1: the user starts to shoot the goods shelf in an image shooting mode.

In such a scenario, the user takes a picture of the shelf in a manner of taking one picture at a time. Since only a part of the shelf can be photographed at a time, it is necessary to photograph a plurality of times. In addition, the image captured at a certain time has a certain degree of image overlap with the image captured at the previous time, the image captured at the subsequent time, the image at the corresponding upper position of the shelf, and the image at the corresponding lower position of the shelf. Optionally, the degree of overlap may be set to be greater than or equal to 20% to ensure effective identification and stitching of subsequent images.

Step B1: in the process of shooting the shelf, carrying out shelf edge detection on the shot images in real time, and if the shelf edges are detected in the images, entering the step C1; if no shelf edge is detected in the image, guidance information prompting the user to continue shooting is directly generated, and step B1 is repeatedly executed.

Step C1: the acceleration and the angular velocity of the mobile phone on a space coordinate system (namely an X axis, a Y axis and a Z axis) are respectively calculated through an accelerometer and a gyroscope of the mobile phone, and whether the mobile phone has an upward angle or a downward angle is judged according to the calculation result, so that whether the user has the intention of continuously shooting other parts of the goods shelf is analyzed.

If there is no intention to continue photographing, step D1 is performed, and if there is an intention to continue photographing, step E1 is performed.

Step D1: if the mobile phone keeps a certain angle all the time and has almost no change of direction, the user is shown to finish shooting the whole shelf, the user does not have the intention of continuing shooting, so that the guiding information indicating the end of shooting is generated and can be displayed on the screen of the mobile phone to guide the user. After the execution of step D1 is completed, step F1 is executed.

Step E1: if the direction of the upward shooting or the downward shooting of the mobile phone is suddenly changed, the intention of the user to move the upper part or the lower part of the shooting shelf is shown, so that the guide information for instructing the user to move upwards or downwards and continue shooting is generated, and the guide information can be displayed on the screen of the mobile phone to guide the user.

Optionally, after the guidance information is generated, the posture data of the image acquisition device may be acquired again, and it is determined whether the user performs an operation according to the guidance information according to the posture data, and if the user does not perform an operation according to the guidance information, warning information may be generated to prompt the user; if the user performs an operation in accordance with the instruction of the guidance information, the operation may not be performed.

After the newly acquired image is detected, the process returns to step B1 to continue execution.

Step F1: and finishing shooting, and splicing a plurality of shelf images shot by the user so as to generate a complete image containing a whole shelf.

Through this process, carry out goods shelves edge detection to the image of shooing, if whether have shot goods shelves edge according to testing result analysis user, if shoot goods shelves edge then through acceleration sensor and gyroscope analysis user on the cell-phone whether have the intention of shooing the rest of goods shelves to guide the user to shoot a complete section of goods shelves better according to the intention that analyzes out, guaranteed the shooting quality, guarantee to obtain the complete image information of goods shelves.

Usage scenario two

Referring to fig. 5b, a flowchart of the steps of the shelf image capture method in the second usage scenario is shown.

In the use scenario, the image capturing device may be a mobile phone, a pad, a camera, or the like. By the method, the complete image of the goods shelf can be obtained, and then the commodity information is analyzed, so that goods replenishment or commodity placement position adjustment can be performed according to the commodity information prompt.

Specifically, taking an image acquisition device as a mobile phone and a target object as a shelf as an example, the shelf image acquisition process includes:

step A2: and acquiring a shelf image acquired according to the indication of the first guide information.

Wherein, the goods shelves are used for bearing the weight of commodity. The shelf may be a shelf for displaying products in a place such as a mall or a supermarket, or a shelf for placing products in a warehouse. The shelf image includes information on a part of the shelf.

The first guide information is used for indicating an image acquisition path of the shelf. The image acquisition path is generated by segmenting the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of an overall plan view, a perspective view and a preset shelf virtual model of the shelf.

The shelf structure information can be an overall image of the shelf shot by a user in advance, and the overall image shot in advance is only used for obtaining the shelf structure information, so that the overall image of one or more shelves with different visual angles can be shot, the shelf structure information of the shelf can be conveniently analyzed from the overall image by a server or a client, and an image acquisition path is generated through the shelf structure information.

Of course, the shelf structure information may also be obtained in other manners, for example, virtual models of shelves with different specifications are established in advance, and a user may select a virtual model of a shelf requiring image acquisition in advance and generate an image acquisition path according to the virtual model.

The detailed implementation of the analysis of the shelf structure can be realized by those skilled in the art in any suitable manner or algorithm according to actual needs, including but not limited to connected domain analysis, analysis using neural network models, and the like. The first guiding information may be generated locally, or may be acquired by the client from the server after being generated by the server.

For example, the schematic diagram of the image capturing path is shown in fig. 5c, a dashed line indicated by 001 in the diagram is a dividing path for dividing the shelf, and the generation of the dividing path according to the shelf structure information may be implemented by a server or locally implemented in the image capturing device. When an image acquisition path is generated, different segmentation paths can be generated according to the structure of the same shelf, and a specific segmentation strategy can be preset in a server or a client, for example, linear segmentation, S-shaped segmentation, U-shaped segmentation, rectangular segmentation, spiral segmentation and the like are adopted, or the segmentation paths can be generated according to the information output of the shelf structure through a trained neural network model.

The line indicated at 002 in the figure is the image capturing path corresponding to the first guide information in the segmentation path. Typically the image acquisition path may be part or all of the segmentation path. The image is indicated at 003 in the figure as the shooting area of one image acquisition of the image acquisition device, which covers at least part of the image acquisition path, and the shooting areas corresponding to two adjacent acquisitions have partial coincidence.

Based on the first guiding information, the user can shoot the corresponding part of the shelf along the corresponding path according to the guiding instruction of the user, and obtain the corresponding shelf image.

Step B2: and obtaining an edge detection result of the shelf edge detection of the shelf image.

After the currently photographed shelf image is obtained, shelf edge detection can be performed on the shelf image. The detection can be executed locally in the image acquisition equipment, and an edge detection result is directly obtained; or the goods shelf image is sent to the server side, the server side carries out goods shelf edge detection, and the edge detection result is sent to the image acquisition equipment.

If the shelf edge detection is performed locally on the image acquisition equipment, a trained lightweight neural network model for performing the shelf edge detection can be adopted for detection, so that the calculated amount is reduced, and the calculation capability of the image acquisition equipment can meet the detection requirement.

If the goods shelf edge detection is carried out at the server side, a trained neural network model with a deep depth for carrying out the goods shelf edge detection can be adopted for carrying out the detection, so that the detection precision is improved.

If the obtained edge detection result indicates that the shelf image includes a shelf edge, executing step C2; and otherwise, generating fourth guide information which indicates that the camera is moved for a certain distance along the image acquisition path to continue shooting.

Step C2: and if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition.

Wherein the new image acquisition path may be a part of the segmentation path, which may be determined based on the actual detection result and the previous path segmentation result, e.g. the new image acquisition path is the path indicated by the dashed line in the lower part of fig. 5 c. In a subsequent step, the user can move the image capturing device to a position where the viewing position corresponds to the new image capturing path (e.g., the dashed line capturing area in fig. 5 c), and continue capturing.

One possible implementation of step C2 includes: if the edge detection result indicates that the shelf images comprise shelf edges, performing commodity information identification on the acquired result images generated by all the acquired shelf images and acquiring a commodity information result; and according to the commodity information result, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition.

The acquired result images generated by all the acquired shelf images can be generated locally in the image acquisition equipment, or the shelf images can be sent to the server after one shelf image is acquired each time, and the server generates the acquired result images and sends the acquired result images to the image acquisition equipment.

The process of generating the acquisition result image locally at the image acquisition device may be: and acquiring an acquisition result image generated by splicing all the acquired shelf images.

For example, the image capture device superimposes the overlapping portions of the two images based on the overlapping portions in the shelf images to form a stitched capture result image. If the right side of the image 1 and the left side of the image 2 have overlapped parts, the overlapped parts of the image 1 and the image 2 are overlapped to form an acquisition result image.

For the convenience of viewing by the user, a preview box (as shown at 005 in fig. 5 d) may be configured in the display interface, and the spliced acquisition result images are shown in the preview box.

The acquiring of the second guide information indicating a new image acquisition path or the acquiring of the third guide information indicating ending of acquisition according to the commodity information result includes: if the commodity information result indicates that all commodities of the shelf are not contained in the acquisition result image, second guide information indicating switching of shooting in the shooting path is acquired; or if the commodity information result indicates that all commodities of the shelf are contained in the acquisition result image, acquiring third guide information indicating that shooting is finished.

This kind of mode can generate accurate guide information to guide the user to carry out a lot of collection to goods shelves image, can guarantee to include clear commodity information in the goods shelves image of every collection, can supply discernment, thereby solve current goods shelves size overlength, acquire the problem that goods shelves whole image can make commodity information undersize, can not discern through once gathering.

Another possible implementation of step C2 includes: if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring attitude data of image acquisition equipment; and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition according to the attitude data.

Wherein the pose data comprises acceleration information and/or angular velocity information of the image acquisition device in a spatial coordinate system. Whether the user has the intention of continuing to acquire the image or not can be determined according to the posture data, and then the direction of the intention shooting can be determined when the user has the intention of continuing to acquire the image, so that corresponding second guide information is generated; the third guidance information may be generated when there is no intention to continue image acquisition.

In this way, accurate guidance information can be generated to ensure that a shelf image including the commodity information available for identification can be acquired, and thus that corresponding processing can be performed in subsequent steps according to the identified commodity information.

Optionally, during the image acquisition, before the other acquisition after the first acquisition, it may also be performed that: and acquiring a reserved area corresponding to the current image acquisition path from the latest acquired shelf image, and displaying the reserved area in a set area of a display interface so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area.

Through showing the reserved area in the set area, the user can conveniently align the reserved area with the corresponding position of the goods shelf when next image acquisition is carried out, so that the overlapped part of the goods shelf images acquired by the user twice is ensured to be spliced sufficiently, and the situation that the overlapped part leads to the excessive acquisition times required by acquiring the complete information of the goods shelf due to the excessive overlapping part can be prevented.

The reserved area is as shown at 006 in fig. 5d, and the reserved area in the newly acquired shelf image and the set area in the display interface are determined according to the image acquisition path.

For example, for the most recently acquired shelf image, the next shelf image is acquired by moving a distance to the right along the image acquisition path, and the reserved area is the rightmost partial area of the most recently acquired shelf image, which may be 1/6 to 1/5 of the total area. Correspondingly, the setting area is the leftmost partial area of the display interface.

For another example, if the latest shelf image is acquired by moving down to a new image acquisition path, the reserved area is the lowermost partial area of the latest shelf image, and the area of the partial area may be 1/6 to 1/5 of the total area. Correspondingly, the setting area is the uppermost partial area of the display interface.

Therefore, when the user carries out the next image acquisition operation, the reserved area in the display interface is aligned with the corresponding area in the actual shelf, so that the acquired goods image and the previous goods image have enough overlapped parts to determine the position relation between the two goods images, and the phenomenon that the overlapped parts are too much to cause a large amount of useless data is avoided.

Optionally, the identifying step may also be performed at any suitable time after the acquisition of the acquisition result image, namely: and carrying out commodity information identification and/or commodity position identification on the acquired result image, and obtaining a commodity information result and/or a commodity position result.

For example, the identification step may be performed after each image capturing operation and after a captured result image is generated from a captured shelf image, or may be performed after a captured result image including shelf integrity information is acquired.

Preferably, in order to improve efficiency, the identifying step is performed after the acquisition result image containing the shelf integrity information is acquired.

The commodity information identification can be executed at a server side or locally.

When the server side executes, the server side obtains an acquisition result image sent by the image acquisition equipment, or obtains a shelf image sent by the image acquisition equipment and splices the shelf image to form an acquisition result image, and then the neural network model which is trained and can be used for identifying commodity information is used for identifying to obtain a commodity information result.

When the system is executed locally, the image acquisition equipment can splice acquisition result images directly according to the shelf images or obtain the acquisition result images from the server side, and a trained neural network model capable of identifying commodity information is used for identification to obtain a commodity information result.

Similarly, the goods location identification may be performed at the server or locally. During recognition, a trained neural network model capable of recognizing the commodity position can be used for recognition, and a commodity position result is obtained.

Optionally, after performing commodity information identification and/or commodity position identification on the acquired result image and obtaining a commodity information result and/or a commodity position result, the following steps may be further performed: and analyzing the commodity information result and/or the commodity position result, and generating an analysis result corresponding to the analysis operation.

The analysis result includes at least one of: commodity selling information, commodity display information, commodity quantity information and commodity replenishment state information.

Different analysis operations may be performed in order to obtain different analysis results.

For example, if the analysis result includes the commodity selling information, the commodity information result and the commodity position result may be analyzed to determine the remaining commodities on the shelf and the commodities at each placement position on the shelf, and further analyze the commodities at the vacant positions, thereby determining the commodity selling information.

For example, if the analysis result includes product display information, the product information result may be analyzed to specify products placed at each placement position on the shelf, and the product display information may be specified.

For another example, if the analysis result includes the commodity quantity information, the commodity information result and the commodity position result may be analyzed to determine the commodities at each placement position, and the commodity quantity information may be determined according to the number of placement positions.

For another example, if the analysis result includes the commodity replenishment state information, the commodity information result and the commodity position result may be analyzed to determine the commodity to be replenished and the corresponding replenishment position.

Optionally, when receiving the termination of the acquisition operation by the user, the following steps may also be performed: and determining whether all goods of the shelf are contained in the acquired result image generated according to all the acquired shelf images.

Because the user may have an unexpected situation in the image acquisition process to terminate the image acquisition, when an operation (such as quitting the operation or ending the acquisition operation) that the user instructs to terminate the image acquisition is obtained, it is determined whether the shelf has been completely acquired, that is, whether all the commodities of the shelf are included in the acquired result images generated by all the acquired shelf images.

If the acquisition result image contains all the commodities on the shelf, the complete acquisition is indicated, the acquisition result image can be stored, and the acquisition is stopped.

If the acquired result image does not contain all goods on the shelf, indicating that complete acquisition is not performed, the acquired result image and related acquisition information (such as an image acquisition path and the like) can be saved, and the user is prompted to have prompt information of a part which is not uploaded (or not updated), so that the user is informed that image acquisition can be continued at a proper time.

Therefore, the acquired shelf images can be fully utilized, timely and intelligently analyzed, the analysis result is used for carrying out corresponding processing according to the requirement, and if the requirement of replenishment is determined according to the analysis result, a replenishment prompt can be generated, and the type of goods to be replenished is indicated.

Usage scenario three

Referring to fig. 5e, it shows a flow chart of the steps of the commodity information processing method in the third usage scenario.

In the use scene, taking the image acquisition device as a mobile phone as an example, the method comprises the following steps:

step A3: and acquiring image data of the shelf according to the acquired first guiding information.

Wherein the first guiding information is used for indicating an image acquisition path of the shelf. The first guidance information may be generated in the manner described in the usage scenario two, or may be generated in another manner, which is not limited in this usage scenario.

Step B3: and identifying the image data, and obtaining commodity information on the shelf and information whether the commodity information contains the edge of the shelf.

The identification may include merchandise information identification and shelf edge identification. The commodity information identification can be carried out by using a neural network model capable of carrying out commodity information identification in the second scene, or by adopting other modes. The shelf edge recognition can be performed by using a neural network model capable of performing shelf edge recognition, or by using other methods.

If the information indicates that the shelf edge is included in the image data, performing step C3; otherwise, guidance information indicating that acquisition is to continue along the image acquisition path may be deactivated or generated.

Step C3: and if the image data contains the information of the edge of the shelf, judging whether all the acquired image data contain all the commodity information according to the commodity information, and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition according to a judgment result.

When judging whether all the acquired image data contain all the commodity information, determining the number of commodity types according to the commodity information, if the number of the commodity types meets the requirement, judging that the result contains all the commodities, and acquiring third guide information indicating the end of acquisition according to the judgment result; and if the number of the commodity types does not meet the requirement, judging that all commodities are not contained, and acquiring second guide information indicating a new image acquisition path according to the judgment result.

The subsequent method can also execute other steps according to the commodity information, such as generating replenishment prompt information and the like. The commodity information processing method of the use scene can realize the processing of commodity information on the goods shelf, thereby meeting the requirements of replenishment prompt, commodity position change prompt and the like.

Usage scenario four

Referring to fig. 5f, a flowchart illustrating the steps of the shelf image capture method in the present usage scenario four is shown.

In the use scene, the image acquisition equipment is a mobile phone, and the method comprises the following steps:

step A4: and displaying first acquisition prompt information of goods on the goods shelf.

The first acquisition prompt information is used for indicating an acquisition position when the image acquisition is carried out on the goods on the shelf along the image acquisition path. The first collection prompt information may be determined according to the image collection path indicated by the first guide information, for example, for the last collection position, a new collection position is determined by moving a certain distance on the image collection path, and the first collection prompt information is generated according to the new collection position. The first guidance information may be as described in usage scenario two.

Step B4: and acquiring an image for image acquisition according to the first acquisition prompt information, and identifying the acquired image.

The acquired images can be identified differently according to different needs. For example, shelf edge recognition, product information recognition, and the like are performed. The specific identification method can be the same as the method described in the aforementioned usage scenario, and therefore, the detailed description thereof is omitted here.

If the identification result indicates that the image includes a shelf edge, executing step C4; otherwise, the first acquisition prompt message is updated according to the image acquisition path, and the step A4 is returned to continue the execution.

Step C4: and if the identification result indicates that the image comprises the shelf edge, displaying second acquisition prompt information for indicating a new image acquisition path and indicating to continue image acquisition.

And if the shelf edge is included, determining a new image acquisition path, and generating second acquisition prompt information indicating the new image acquisition path to prompt the user to switch the image acquisition path for continuous acquisition. The process of determining the new image acquisition path may be the same as the aforementioned usage scenario, and is therefore not described in detail.

By the shelf image acquisition method of the use scene, complete and accurate shelf images and commodity information on the shelves can be obtained, so that the commodity information can be analyzed to perform replenishment prompt, commodity position change prompt and the like.

Usage scenario five

Referring to fig. 5g, a schematic structural diagram of a presentation interface of the client in the present usage scenario five is shown.

In this usage scenario, the client includes a presentation interface. The display interface is used for displaying first acquisition prompt information, and the first acquisition prompt information is used for indicating image acquisition of the target object along an image acquisition path; the display interface is further configured to display second acquisition prompt information (as shown at 007 in fig. 5 g), where the second acquisition prompt information indicates that image acquisition is performed on the target object along a new image acquisition path when an edge of the target object is included in the acquired image.

The first collection prompt information can be generated in the manner described in the fourth scenario and displayed through the display interface. The second acquisition prompt may be determined based on the new image acquisition path. For example, for the latest acquisition position, the new acquisition position is determined by moving a certain distance on the new image acquisition path, and second acquisition prompt information is generated according to the new acquisition position and is displayed through the display interface.

The client can display the first acquisition prompt information and the second acquisition prompt information so as to prompt a user to acquire images, so that the quality of the acquired images is improved, and the high-quality complete images of the target object can be acquired.

Optionally, the target object comprises at least one of: goods shelves, parking lots, seats of venues. For the parking lot, the method can acquire the complete image of the parking lot, and further can analyze the vehicle information in the image. For the stadium seat, the method can be used for collecting the complete image of the stadium seat, further analyzing the use condition of the seat, further calculating the seat-in rate and the like. For the goods shelf, the method can acquire the complete image of the goods shelf, and further can analyze the vehicle information, the seat information or the commodity information and the like in the goods shelf, so that the subsequent processing can be performed.

Usage scenario six

Referring to fig. 5h, a flowchart illustrating the steps of the commodity information processing method in the sixth usage scenario is shown.

In this usage scenario, the method includes:

step A5: image data of the shelf is collected.

The image data of the shelf can be collected through image collecting equipment, and the image collecting equipment can be a mobile phone and the like. The acquisition of the image data of the shelf can be realized by adopting any one of the above-mentioned use scenes from one to five.

Step B5: and processing the image data, and identifying to obtain the commodity information on the shelf.

The image data may be processed differently according to different needs, for example, the image is processed by using a trained neural network model capable of recognizing commodity information in the image, and commodity information on a shelf is acquired, and the commodity information may include commodity name information, category information, and the like.

Step C5: and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

The commodity statistical information may include commodity quantity information, commodity category quantity, commodity quantity of each category, and the like.

By the method, the image data with high quality and containing all information of the goods shelf can be obtained, and further the image data can be analyzed so as to obtain the commodity statistical information, so that the replenishment prompt and the like can be conveniently carried out according to the commodity statistical information.

Usage scenario seven

Referring to fig. 5i, a flow chart illustrating the steps of the commodity information processing method in the present usage scenario seven is shown.

In this usage scenario, the method includes:

step A6: and calling an image acquisition device of the client to shoot image data of the goods shelf in response to shooting operation initiated by the user.

The user can call the image acquisition device of the client through the server, or directly initiate shooting operation at the client and call the image acquisition device.

The manner of acquiring the image data may be any one of the manners described in the usage scenarios one to five, which are not limited in this usage scenario.

Step B6: and processing the image data, and identifying to obtain the commodity information on the shelf.

The manner of acquiring the commodity information may be the same as or different from the aforementioned usage scenario six.

Step C6: and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

Usage scenario eight

Referring to fig. 5j, a schematic flow chart of steps of the processing method for commodity replenishment in the present usage scenario eight is shown.

In this usage scenario, the method comprises:

step A7: and calling the image acquisition device to shoot image data of the goods shelf in response to the replenishment operation initiated by the user.

The user can initiate the replenishment operation through the client, the client directly calls the image data of the goods shelf shot by the image acquisition device, or the client sends the replenishment operation to the server, and the server calls the image data of the goods shelf shot by the image acquisition device.

The manner in which the image data is captured may be captured as in any of the manners using scenes one through five.

Step B7: and carrying out identification processing on the image data to identify and obtain commodity information on the shelf.

Step C7: and determining the goods to be replenished according to the goods information on the goods shelf.

For example, the remaining commodities are determined according to the commodity information, and commodities except the remaining commodities in the preset commodity information are determined as commodities to be restocked.

Optionally, the method further comprises:

step D7: and generating and displaying replenishment prompt information for prompting replenishment of the goods to be replenished according to the goods to be replenished.

The person skilled in the art can generate the replenishment prompting message in a suitable manner, for example, directly according to the name of the goods to be replenished.

By the method, the complete and high-quality shelf image can be obtained, and then the commodity information of the shelf is obtained, so that the commodity to be supplemented is determined according to the commodity information, the commodity to be supplemented can be automatically obtained in a mode of shooting the shelf image, the shelf is not required to be manually counted one by a user, the user can be rapidly prompted to supplement the commodity by generating the supplement prompt information, and convenience is improved.

Usage scenario nine

Referring to fig. 5k, a schematic diagram of information interaction among a user, an image capturing device and a server in a usage scenario nine is shown.

In this usage scenario, the replenishment process includes:

and the image acquisition equipment receives the trained commodity identification model issued by the server. When receiving a shooting start instruction of a user, the image acquisition device acquires the shelf image by the image acquisition method of the first embodiment to the fourth embodiment, and acquires an acquisition result image of the shelf. And carrying out commodity information processing on the acquired result image by using a commodity identification model, and displaying the recommended commodity according to the processing result. After receiving the selection operation of the user on the recommended commodity, determining the commodity to be replenished according to the selected commodity, and submitting a replenishment request to the server so as to generate a replenishment order at the server.

In addition, after the image acquisition device obtains the processing result of the commodity information processing, the processing result can be sent to the server, so that the server continues to train the initial commodity identification model, the trained initial commodity identification model is compressed periodically or according to other conditions, and the compression result is sent to the image acquisition device.

The goods replenishment process can ensure the collection quality of goods shelf images, and further ensure the quality of commodity information processing, thereby realizing reliable automatic goods replenishment.

EXAMPLE five

Referring to fig. 6, a flowchart illustrating steps of an image capturing method according to a fifth embodiment of the present invention is shown.

The image acquisition method of the embodiment comprises the following steps:

step S602: in the process of image acquisition of a target object, attitude data of an image acquisition device is acquired.

The pose data is used to indicate the pose in which the image acquisition device is held, e.g. horizontally held, vertically held, has an upward tilt or has a downward tilt, etc. The image acquisition intention of the user can be determined according to the held gesture. For example, in the image capturing process, the user intends to perform continuous capturing along the current image capturing path, and the image capturing apparatus is generally held vertically; however, if the user intends to switch to a new image capturing path for a subsequent image capturing, the image capturing apparatus is generally held with an upward inclination or with a downward inclination.

The pose data of the image capturing device includes, but is not limited to, acceleration information and/or angular velocity information of the image capturing device in a spatial coordinate system. It may also include distance information to the target object, etc.

The person skilled in the art may obtain the attitude data of the image capturing device in a suitable manner, for example, obtaining acceleration information by an acceleration sensor and obtaining angular velocity information by a gyroscope.

Step S604: and generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information.

For example, when the attitude data includes acceleration information and/or angular velocity information, step S604 may be implemented as: determining the current posture of the image acquisition equipment according to the acceleration information and/or the angular velocity information; and generating guide information for indicating the user to move to the direction matched with the current posture so as to collect continuous images according to the current posture.

In the first case, when the guidance information instructing the user to move to the direction matched with the current posture for continuous image acquisition is generated according to the current posture, if the current posture meets a preset path conversion condition, fifth guidance information for guiding the user to convert the current image acquisition path into a new image acquisition path matched with the current posture and perform continuous image acquisition along the new image acquisition path is generated.

And if the current posture is that the image acquisition equipment has a downward inclination angle or an upward inclination angle, determining that the current posture meets a preset path conversion condition, and generating fifth guide information, wherein the fifth guide information guides a user to convert the current image acquisition path into a new image acquisition path matched with the current posture and carry out continuous image acquisition along the new image acquisition path. And if the current posture has a downward inclination angle, converting the new image acquisition path into an image acquisition path below the current image acquisition path, and generating fifth guide information with the content of 'please move downwards and continue to shoot'.

The new image acquisition path generated may be different depending on the structure of the target object. A person skilled in the art may generate the image capturing path in any suitable manner as needed, for example, a new image capturing path is generated according to a preset image capturing path generating policy, or the image capturing path generating manner in the foregoing embodiments is adopted.

In the second case, if the current posture does not meet the preset path conversion condition, sixth guidance information for guiding the user to perform continuous image acquisition along the current image acquisition path is generated.

For example, if the current posture is that the image capturing device is held vertically, the current posture does not meet the preset path conversion condition, and then guidance information instructing the user to move along the current image capturing path and perform continuous image capturing is generated.

According to the embodiment, in the image acquisition process, the posture data of the image acquisition equipment is acquired, and then the guiding information which indicates that the user moves to the direction matched with the current posture to perform continuous image acquisition can be generated according to the posture data, so that the user can be better guided to perform image acquisition on the target object.

EXAMPLE six

Referring to fig. 7, a flowchart illustrating steps of an image acquisition method according to a sixth embodiment of the present invention is shown.

In this embodiment, the image capturing method includes the aforementioned steps S602 to S604.

Wherein the method further comprises:

step S604 a: and acquiring the image of the target object acquired by the image acquisition equipment in real time.

The image of the target object may be an image captured by a user using the image capturing apparatus according to the guidance information. For example, if the target object is a shelf, the image may be an image containing a portion of the shelf.

Step S604 b: and carrying out edge detection on the acquired image to obtain a detection result.

The edge detection may be performed on the image in any suitable manner, for example, by using a trained neural network model for edge detection to perform edge detection on the image and obtain a detection result. Alternatively, the edge detection method for the image in any of the foregoing embodiments may be adopted.

The detection result may indicate that the captured image includes an edge of the target object, or does not include an edge of the target object, or the like.

In the case of obtaining the detection result, the step S604 includes generating corresponding guidance information according to the posture data and the detection result, and guiding the user to perform continuous image acquisition on the target object through the guidance information.

The user may shake during shooting, so that the posture data of the image acquisition device indicates that the posture of the image acquisition device changes, the generated guide information is ensured to be accurate in order to avoid the influence of the shake on the generated guide information, the edge detection is performed on the acquired image, the guide information is generated by combining the posture data and the detection result, and the generated guide information can be more accurate.

For example, in the first case, if the current posture meets a preset path conversion condition and the detection result indicates that the edge of the target object is detected, fifth guidance information for guiding the user to convert the current image capturing path into a new image capturing path matched with the current posture and perform continuous image capturing along the new image capturing path is generated.

When it is determined that the current pose meets a preset path conversion condition according to the pose data, for example, the current pose has a downward inclination, and the detection result indicates that the edge of the target object is detected, it indicates that the user wishes to continue to acquire images of other portions of the target object downward, and therefore, fifth guide information may be generated that guides the user to convert the current image acquisition path into a new image acquisition path that matches the current pose and to continue image acquisition along the new image acquisition path.

For another example, in the second case, if the current posture does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, sixth guidance information for guiding the user to perform continuous image acquisition along the current image acquisition path is generated.

When it is determined that the current gesture does not meet the preset path conversion condition according to the gesture data, for example, the current gesture is a vertical holding, and the detection result indicates that the edge of the target object is not detected, it indicates that the user wishes to continue to acquire the image of the other portion of the target object along the current image acquisition path, and therefore, sixth guidance information for guiding the user to perform continuous image acquisition along the current image acquisition path may be generated.

Optionally, in this embodiment, the image acquisition method may further include:

step S606: and generating seventh guide information for guiding the user to stop image acquisition according to the attitude data and the detection result.

For example, if the current posture determined according to the posture data does not meet the preset path conversion condition and the detection result indicates that the edge of the target object is detected, seventh guidance information for guiding the user to stop continuing image acquisition is generated.

In addition, the collected image can be subjected to edge detection, so that whether the user is guided to continue shooting is determined, and the intelligence is better.

EXAMPLE seven

Referring to fig. 8, a block diagram of an image capturing apparatus according to a seventh embodiment of the present invention is shown.

The image acquisition apparatus of the present embodiment includes: a detection module 802, configured to obtain a detection result of performing real-time target object edge detection on an acquired image, where the acquired image includes partial image information of a target object; a first obtaining module 804, configured to obtain gesture data of an image capturing device that captures the image if the detection result indicates that the edge of the target object is detected in the image; a generating module 806, configured to generate corresponding guiding information according to the posture data, and guide a user to perform subsequent image acquisition on the target object through the guiding information, so as to form complete image information of the target object by using the acquired multiple images.

Example eight

Referring to fig. 9, a block diagram of an image capturing apparatus according to an eighth embodiment of the present invention is shown.

The image acquisition apparatus of the present embodiment includes: a detection module 902, configured to obtain a detection result of performing real-time target object edge detection on an acquired image, where the acquired image includes partial image information of a target object; a first obtaining module 904, configured to obtain pose data of an image capturing device that captures the image if the detection result indicates that the edge of the target object is detected in the image; a generating module 906, configured to generate corresponding guiding information according to the posture data, and guide a user to perform subsequent image acquisition on the target object through the guiding information, so as to form complete image information of the target object by using the acquired multiple images.

Optionally, the apparatus further comprises: a second obtaining module 908, configured to obtain a lightweight neural network model dynamically issued to the image acquisition device and used for performing edge detection on the target object before the detection result of performing real-time edge detection on the target object on the acquired image is obtained; the detection module 902 is configured to perform real-time target object edge detection on the acquired image by using the lightweight neural network model, so as to obtain a detection result.

Optionally, the first obtaining module 904 is configured to obtain acceleration information and/or angular velocity information of the image capturing apparatus in a spatial coordinate system; the generating module 906 includes: the first determining module 9061 is configured to determine the current posture of the image acquisition device according to the acceleration information and/or the angular velocity information; and the information generation module 9062 is configured to generate, according to the current posture, guidance information for instructing the user to move to a direction matched with the current posture to perform continuous image acquisition.

Optionally, the apparatus further comprises: a stitching module 910, configured to stitch the acquired multiple images to obtain a complete image including complete image information of the target object.

Optionally, the splicing module 910 includes: a second determining module 9101, configured to determine, from the multiple acquired images, multiple groups of images having an image coincidence relationship, where each group of images includes two images; a complete image obtaining module 9102, configured to splice the multiple acquired images according to the image coincidence relationship, and obtain a complete image including complete image information of the target object according to a splicing result.

Optionally, the second determining module 9101 includes: the characteristic extraction module is used for extracting the characteristics of each image in the plurality of acquired images to obtain characteristic points corresponding to each image; and the matching module is used for matching any two images according to the characteristic points of the two images and determining the multiple groups of images with the image coincidence relation based on the matching result.

The image acquisition device of this embodiment is used to implement the corresponding image acquisition method in the foregoing method embodiments, and has the beneficial effects of the corresponding method embodiments, which are not described herein again.

Example nine

Referring to fig. 10, a schematic structural diagram of an electronic device according to a ninth embodiment of the present invention is shown, and the specific embodiment of the present invention does not limit the specific implementation of the electronic device.

As shown in fig. 10, the electronic device may include: a processor (processor)1002, a Communications Interface 1004, a memory 1006, and a Communications bus 808.

Wherein:

the processor 1002, communication interface 1004, and memory 1006 communicate with each other via a communication bus 1008.

A communication interface 1004 for communicating with other electronic devices such as a terminal device or a server.

The processor 1002 is configured to execute the program 1010, and may specifically perform relevant steps in the above-described embodiment of the image capturing method.

In particular, the program 1010 may include program code that includes computer operating instructions.

The processor 1002 may be a central processing unit CPU, or an application Specific Integrated circuit asic, or one or more Integrated circuits configured to implement an embodiment of the present invention. The electronic device comprises one or more processors, which can be the same type of processor, such as one or more CPUs; or may be different types of processors such as one or more CPUs and one or more ASICs.

The memory 1006 is used for storing the program 1010. The memory 1006 may comprise high-speed RAM memory, and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.

The program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: obtaining a detection result of real-time target object edge detection on an acquired image, wherein the acquired image comprises partial image information of a target object; if the detection result indicates that the edge of the target object is detected in the image, acquiring attitude data of image acquisition equipment for acquiring the image; and generating corresponding guide information according to the attitude data, and guiding a user to perform continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using the acquired multiple images.

In an optional implementation manner, the program 1010 is further configured to enable the processor 1002 to obtain a lightweight neural network model dynamically issued to the image capturing device and used for performing edge detection on the target object before obtaining a result of performing edge detection on the captured image for the target object in real time; and when the detection result of carrying out real-time target object edge detection on the acquired image is obtained, carrying out real-time target object edge detection on the acquired image by using the lightweight neural network model to obtain the detection result.

In an alternative embodiment, the program 1010 is further configured to, when the processor 1002 acquires the posture data of the image capturing device capturing the image, acquire acceleration information and/or angular velocity information of the image capturing device in a spatial coordinate system; when corresponding guide information is generated according to the attitude data and the user is guided to perform continuous image acquisition on the target object through the guide information, determining the current attitude of the image acquisition equipment according to the acceleration information and/or the angular velocity information; and generating guide information for indicating the user to move to the direction matched with the current posture so as to collect continuous images according to the current posture.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002 to stitch the acquired plurality of images to obtain a complete image including the complete image information of the target object when the acquired plurality of images are used to form the complete image information of the target object.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002 to determine multiple sets of images having an image registration relationship from the multiple acquired images when the multiple acquired images are spliced to obtain a complete image containing complete image information of the target object, where each set of images includes two images; and splicing the plurality of acquired images according to the image coincidence relation, and acquiring a complete image containing complete image information of the target object according to a splicing result.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002, when determining multiple sets of images having an image coincidence relationship from the acquired multiple images, to perform feature extraction on each of the acquired multiple images to obtain a feature point corresponding to each image; and matching any two images according to the characteristic points of the two images, and determining the multiple groups of images with the image coincidence relation based on the matching result.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: acquiring a shelf image acquired according to the indication of first guide information, wherein the shelf is used for bearing commodities, and the first guide information is used for indicating an image acquisition path of the shelf; obtaining an edge detection result of the shelf edge detection of the shelf image; and if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition.

In an optional implementation, the program 1010 is further configured to cause the processor 1002 to obtain first guiding information before obtaining a shelf image collected according to an instruction of the first guiding information, where the first guiding information is guiding information corresponding to the image collecting path, the image collecting path is a path generated by dividing the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of an overall plan view, a perspective view, and a preset shelf virtual model of the shelf.

In an optional implementation, when the edge detection result indicates that the shelf images include shelf edges, the program 1010 is further configured to enable the processor 1002, when acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating that acquisition is finished, if the edge detection result indicates that the shelf images include shelf edges, perform commodity information identification on acquisition result images generated by all the acquired shelf images, and acquire a commodity information result; and according to the commodity information result, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition.

In an optional implementation, the program 1010 is further configured to, when acquiring second guide information indicating a new image capturing path or third guide information indicating that capturing is finished according to the product information result, if the product information result indicates that all the products on the shelf are not included in the captured result image, acquire second guide information indicating switching of capturing in the capturing path; or if the commodity information result indicates that all commodities of the shelf are contained in the acquisition result image, acquiring third guide information indicating that shooting is finished.

In an optional embodiment, when the edge detection result indicates that the shelf image includes a shelf edge, the program 1010 is further configured to enable the processor 1002, when acquiring the second guide information indicating a new image capturing path or acquiring the third guide information indicating that capturing is finished, to acquire the pose data of the image capturing apparatus if the edge detection result indicates that the shelf image includes a shelf edge; and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition according to the attitude data.

In an alternative embodiment, the pose data includes acceleration information and/or angular velocity information of the image acquisition device in a spatial coordinate system.

In an alternative embodiment, the program 1010 is further configured to enable the processor 1002 to obtain a reserved area corresponding to the current image capturing path from the latest captured shelf image and display the reserved area in a set area of a display interface, so as to indicate an image capturing alignment position of a next image capturing operation through the reserved area.

In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to obtain the captured result image generated by stitching all the captured shelf images.

In an optional embodiment, the program 1010 is further configured to enable the processor 1002 to perform commodity information identification and/or commodity position identification on the acquired result image, and obtain a commodity information result and/or a commodity position result; and analyzing the commodity information result and/or the commodity position result, and generating an analysis result corresponding to the analysis operation.

In an alternative embodiment, the analysis results include at least one of: commodity selling information, commodity display information, commodity quantity information and commodity replenishment state information.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: acquiring image data of a shelf according to acquired first guide information, wherein the first guide information is used for indicating an image acquisition path of the shelf; identifying the image data, and acquiring commodity information on the shelf and information whether the commodity information contains the edge of the shelf; and if the image data contains the information of the edge of the shelf, judging whether all the acquired image data contain all the commodity information according to the commodity information, and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition according to a judgment result.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: displaying first acquisition prompt information of the goods on the shelf, wherein the first acquisition prompt information is used for indicating an acquisition position when the goods on the shelf are subjected to image acquisition along an image acquisition path; acquiring an image for image acquisition according to the first acquisition prompt information, and identifying the acquired image; and if the identification result indicates that the image comprises the shelf edge, displaying second acquisition prompt information for indicating a new image acquisition path and indicating to continue image acquisition.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: collecting image data of a shelf; processing the image data, and identifying to obtain commodity information on the shelf; and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: calling an image acquisition device of a client to shoot image data of a shelf in response to shooting operation initiated by a user; processing the image data, and identifying to obtain commodity information on the shelf; and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: calling an image acquisition device to shoot image data of the goods shelf in response to a replenishment operation initiated by a user; carrying out identification processing on the image data to obtain commodity information on the goods shelf; and determining the goods to be replenished according to the goods information on the goods shelf.

In an optional implementation manner, the program 1010 is further configured to enable the processor 1002 to generate and display replenishment prompting information for prompting replenishment of the goods to be replenished according to the goods to be replenished.

Alternatively, the first and second electrodes may be,

the program 1010 may be specifically configured to cause the processor 1002 to perform the following operations: acquiring attitude data of image acquisition equipment in the process of acquiring an image of a target object; and generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information.

In an alternative embodiment, the attitude data includes acceleration information and/or angular velocity information of the image acquisition device in a spatial coordinate system; the program 1010 is further configured to enable the processor 1002 to determine a current posture of the image capturing device according to the acceleration information and/or the angular velocity information when generating corresponding guidance information according to the posture data and guiding a user to perform continuous image capturing on the target object through the guidance information; and generating guide information for indicating the user to move to the direction matched with the current posture so as to collect continuous images according to the current posture.

In an optional implementation, the program 1010 is further configured to, when generating, according to the current posture, guidance information for instructing the user to move to a direction matched with the current posture for performing continuous image acquisition, if the current posture meets a preset path conversion condition, generate fifth guidance information for guiding the user to convert the current image acquisition path into a new image acquisition path matched with the current posture and perform continuous image acquisition along the new image acquisition path; and if the current posture does not accord with the preset path conversion condition, generating sixth guide information for guiding the user to carry out continuous image acquisition along the current image acquisition path.

In an alternative embodiment, the program 1010 is further configured to cause the processor 1002 to acquire an image of a target object acquired by the image acquisition device in real time; carrying out edge detection on the acquired image to obtain a detection result; and the program 1010 is further configured to enable the processor 1002 to generate corresponding guidance information according to the posture data, and when guiding the user to perform the subsequent image acquisition on the target object through the guidance information, generate corresponding guidance information according to the posture data and the detection result, and guide the user to perform the subsequent image acquisition on the target object through the guidance information.

In an optional implementation manner, the program 1010 is further configured to enable the processor 1002, when generating corresponding guidance information according to the pose data and the detection result, and guiding a user to perform continuous image acquisition on the target object through the guidance information, if the current pose meets a preset path conversion condition and the detection result indicates that an edge of the target object is detected, generate fifth guidance information for guiding the user to convert the current image acquisition path into a new image acquisition path matched with the current pose and perform continuous image acquisition along the new image acquisition path; and if the current posture does not accord with the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, generating sixth guide information for guiding the user to carry out continuous image acquisition along the current image acquisition path.

For specific implementation of each step in the program 1010, reference may be made to corresponding steps and corresponding descriptions in units in the above embodiments of the image acquisition method, which are not described herein again. It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described devices and modules may refer to the corresponding process descriptions in the foregoing method embodiments, and are not described herein again.

Through the electronic equipment of the embodiment, the edge of the target object is detected in real time on the acquired image, when the acquired image contains the edge of the target object, the attitude data of the image acquisition equipment is acquired, and then the corresponding guide information is generated according to the attitude data, so that the user is guided to carry out image acquisition in a standard manner through the guide information, and the aims of finally completing the image acquisition of a plurality of parts contained in the whole target object, avoiding omission and obtaining the complete image of the target object are fulfilled.

It should be noted that, according to the implementation requirement, each component/step described in the embodiment of the present invention may be divided into more components/steps, and two or more components/steps or partial operations of the components/steps may also be combined into a new component/step to achieve the purpose of the embodiment of the present invention.

The above-described method according to an embodiment of the present invention may be implemented in hardware, firmware, or as software or computer code storable in a recording medium such as a CD ROM, a RAM, a floppy disk, a hard disk, or a magneto-optical disk, or as computer code originally stored in a remote recording medium or a non-transitory machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein may be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware such as an ASIC or FPGA. It will be appreciated that the computer, processor, microprocessor controller or programmable hardware includes memory components (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the image acquisition methods described herein. Further, when a general-purpose computer accesses code for implementing the image capture methods shown herein, execution of the code transforms the general-purpose computer into a special-purpose computer for performing the image capture methods shown herein.

Those of ordinary skill in the art will appreciate that the various illustrative elements and method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

The above embodiments are only for illustrating the embodiments of the present invention and not for limiting the embodiments of the present invention, and those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of the present invention, so that all equivalent technical solutions also belong to the scope of the embodiments of the present invention, and the scope of patent protection of the embodiments of the present invention should be defined by the claims.

Claims

1. A shelf image acquisition method is characterized by comprising the following steps:

acquiring a shelf image acquired according to the indication of first guide information, wherein the shelf is used for bearing commodities, and the first guide information is used for indicating an image acquisition path of the shelf;

obtaining an edge detection result of the shelf edge detection of the shelf image;

and if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition.

2. The method of claim 1, wherein prior to obtaining the shelf image captured according to the indication of the first guidance information, the method further comprises:

and acquiring the first guide information, wherein the first guide information is guide information corresponding to the image acquisition path, the image acquisition path is a path generated by segmenting the shelf according to the shelf structure information, and the shelf structure information is determined according to at least one of an overall plan view, a perspective view and a preset shelf virtual model of the shelf.

3. The method according to claim 1, wherein the obtaining second guidance information indicating a new image capturing path or obtaining third guidance information indicating ending of capturing if the edge detection result indicates that a shelf edge is included in the shelf image comprises:

if the edge detection result indicates that the shelf images comprise shelf edges, performing commodity information identification on the acquired result images generated by all the acquired shelf images and acquiring a commodity information result;

and according to the commodity information result, acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition.

4. The method according to claim 3, wherein the obtaining of the second guide information indicating a new image capturing path or the third guide information indicating the end of capturing according to the commodity information result comprises:

if the commodity information result indicates that all commodities of the shelf are not contained in the acquisition result image, second guide information indicating switching of shooting in the shooting path is acquired; alternatively, the first and second electrodes may be,

and if the commodity information result indicates that all commodities of the shelf are contained in the acquisition result image, acquiring third guide information indicating that shooting is finished.

5. The method according to claim 1, wherein the obtaining second guidance information indicating a new image capturing path or obtaining third guidance information indicating ending of capturing if the edge detection result indicates that a shelf edge is included in the shelf image comprises:

if the edge detection result indicates that the shelf image comprises a shelf edge, acquiring attitude data of image acquisition equipment;

and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating ending of acquisition according to the attitude data.

6. The method of claim 5, wherein the pose data comprises acceleration information and/or angular velocity information of the image acquisition device in a spatial coordinate system.

7. The method of claim 1, further comprising:

and acquiring a reserved area corresponding to the current image acquisition path from the latest acquired shelf image, and displaying the reserved area in a set area of a display interface so as to indicate the image acquisition alignment position of the next image acquisition operation through the reserved area.

8. The method of claim 1, further comprising:

and acquiring an acquisition result image generated by splicing all the acquired shelf images.

9. The method of claim 8, further comprising:

carrying out commodity information identification and/or commodity position identification on the acquired result image, and obtaining a commodity information result and/or a commodity position result;

and analyzing the commodity information result and/or the commodity position result, and generating an analysis result corresponding to the analysis operation.

10. The method of claim 9, wherein the analysis results include at least one of: commodity selling information, commodity display information, commodity quantity information and commodity replenishment state information.

11. A commodity information processing method, characterized by comprising:

acquiring image data of a shelf according to acquired first guide information, wherein the first guide information is used for indicating an image acquisition path of the shelf;

identifying the image data, and acquiring commodity information on the shelf and information whether the commodity information contains the edge of the shelf;

and if the image data contains the information of the edge of the shelf, judging whether all the acquired image data contain all the commodity information according to the commodity information, and acquiring second guide information indicating a new image acquisition path or acquiring third guide information indicating the end of acquisition according to a judgment result.

12. A shelf image acquisition method is characterized by comprising the following steps:

displaying first acquisition prompt information of the goods on the shelf, wherein the first acquisition prompt information is used for indicating an acquisition position when the goods on the shelf are subjected to image acquisition along an image acquisition path;

acquiring an image for image acquisition according to the first acquisition prompt information, and identifying the acquired image;

and if the identification result indicates that the image comprises the shelf edge, displaying second acquisition prompt information for indicating a new image acquisition path and indicating to continue image acquisition.

13. A client, comprising:

the display interface is used for displaying first acquisition prompt information, and the first acquisition prompt information is used for indicating image acquisition of the target object along an image acquisition path;

the display interface is further configured to display second acquisition prompt information, where the second acquisition prompt information indicates that image acquisition is performed on the target object along a new image acquisition path when the acquired image includes an edge of the target object.

14. The method of claim 13, wherein the target object comprises at least one of: goods shelves, parking lots, seats of venues.

15. A commodity information processing method, characterized by comprising:

collecting image data of a shelf;

processing the image data, and identifying to obtain commodity information on the shelf;

and determining commodity statistical information of the shelf according to the commodity information obtained by identification.

16. A commodity information processing method is characterized by comprising the following steps:

calling an image acquisition device of a client to shoot image data of a shelf in response to shooting operation initiated by a user;

17. A method for processing commodity replenishment is characterized by comprising the following steps:

calling an image acquisition device to shoot image data of the goods shelf in response to a replenishment operation initiated by a user;

carrying out identification processing on the image data to obtain commodity information on the goods shelf;

and determining the goods to be replenished according to the goods information on the goods shelf.

18. The method of claim 17, further comprising:

and generating and displaying replenishment prompt information for prompting replenishment of the goods to be replenished according to the goods to be replenished.

19. An image acquisition method, comprising:

obtaining a detection result of real-time target object edge detection on an acquired image, wherein the acquired image comprises partial image information of a target object;

if the detection result indicates that the edge of the target object is detected in the image, acquiring attitude data of image acquisition equipment for acquiring the image;

and generating corresponding guide information according to the attitude data, and guiding a user to perform continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using the acquired multiple images.

20. The method of claim 19,

the acquiring of the attitude data of the image capturing device capturing the image includes: acquiring acceleration information and/or angular velocity information of the image acquisition equipment in a space coordinate system;

generating corresponding guide information according to the attitude data, and guiding a user to perform continuous image acquisition on the target object through the guide information, wherein the method comprises the following steps:

determining the current posture of the image acquisition equipment according to the acceleration information and/or the angular velocity information;

and generating guide information for indicating the user to move to the direction matched with the current posture so as to collect continuous images according to the current posture.

21. The method of claim 19,

before the obtaining a detection result of the real-time target object edge detection on the acquired image, the method further includes: acquiring a lightweight neural network model which is dynamically issued to the image acquisition equipment and is used for carrying out edge detection on the target object;

the obtaining of the detection result of the real-time target object edge detection on the acquired image comprises: and carrying out real-time target object edge detection on the acquired image by using the lightweight neural network model to obtain a detection result.

22. The method of claim 19, wherein said using the acquired plurality of images to form complete image information of the target object comprises:

and splicing the plurality of acquired images to obtain a complete image containing complete image information of the target object.

23. The method of claim 22, wherein stitching the plurality of acquired images to obtain a complete image containing complete image information of the target object comprises:

determining a plurality of groups of images with an image coincidence relation from a plurality of collected images, wherein each group of images comprises two images;

and splicing the plurality of acquired images according to the image coincidence relation, and acquiring a complete image containing complete image information of the target object according to a splicing result.

24. The method of claim 23, wherein determining sets of images having an image registration relationship from the plurality of acquired images comprises:

extracting the characteristics of each image in the plurality of collected images to obtain characteristic points corresponding to each image;

and matching any two images according to the characteristic points of the two images, and determining the multiple groups of images with the image coincidence relation based on the matching result.

25. An image acquisition method, comprising:

acquiring attitude data of image acquisition equipment in the process of acquiring an image of a target object;

and generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information.

26. The method of claim 25, wherein the pose data comprises acceleration information and/or angular velocity information of the image acquisition device in a spatial coordinate system;

27. The method of claim 26, wherein generating guidance information indicating that a user moves to a direction matched with the current pose for subsequent image acquisition according to the current pose comprises:

if the current posture accords with the preset path conversion condition, generating fifth guide information for guiding the user to convert the current image acquisition path into a new image acquisition path matched with the current posture and continuing image acquisition along the new image acquisition path;

and if the current posture does not accord with the preset path conversion condition, generating sixth guide information for guiding the user to carry out continuous image acquisition along the current image acquisition path.

28. The method of any one of claims 25 to 27, further comprising:

acquiring an image of a target object acquired by the image acquisition equipment in real time; carrying out edge detection on the acquired image to obtain a detection result;

and generating corresponding guide information according to the attitude data and the detection result, and guiding a user to carry out continuous image acquisition on the target object through the guide information.

29. The method of claim 28, wherein generating corresponding guidance information according to the posture data and the detection result, and guiding a user to perform subsequent image acquisition on the target object through the guidance information comprises:

if the current posture accords with a preset path conversion condition and the detection result indicates that the edge of the target object is detected, generating fifth guide information for guiding a user to convert the current image acquisition path into a new image acquisition path matched with the current posture and carrying out continuous image acquisition along the new image acquisition path;

and if the current posture does not accord with the preset path conversion condition and the detection result indicates that the edge of the target object is not detected, generating sixth guide information for guiding the user to carry out continuous image acquisition along the current image acquisition path.

30. An image acquisition apparatus, comprising:

the detection module is used for obtaining a detection result of real-time target object edge detection on an acquired image, wherein the acquired image comprises partial image information of a target object;

a first obtaining module, configured to obtain pose data of an image acquisition device that acquires the image if the detection result indicates that the edge of the target object is detected in the image;

and the generating module is used for generating corresponding guide information according to the attitude data, and guiding a user to carry out continuous image acquisition on the target object through the guide information so as to form complete image information of the target object by using a plurality of acquired images.

31. An electronic device, comprising: the system comprises a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface complete mutual communication through the communication bus;

the memory is used for storing at least one executable instruction which causes the processor to execute the operation corresponding to the shelf image acquisition method according to any one of claims 1-10, or performs an operation corresponding to the commodity information processing method according to claim 11, or performs an operation corresponding to the shelf image capture method according to claim 12, or performs an operation corresponding to the commodity information processing method according to claim 15, or performs an operation corresponding to the commodity information processing method according to claim 16, or the operation corresponding to the processing method of commodity replenishment according to claim 17 or 18 is performed, or to perform operations corresponding to the image acquisition methods according to claims 19-24, or to perform operations corresponding to the image acquisition methods according to claims 25-29.

32. A computer storage medium having stored thereon a computer program which, when being implemented by a processor, implements the shelf image acquisition method according to any one of claims 1 to 10, or implements the merchandise information processing method according to claim 11, or implements the shelf image acquisition method according to claim 12, or implements the merchandise information processing method according to claim 15, or implements the merchandise information processing method according to claim 16, or implements the merchandise replenishment processing method according to claim 17 or 18, or implements the image acquisition method according to claims 19 to 24, or implements the image acquisition method according to claims 25 to 29.