CN111325049A - Commodity identification method and device, electronic equipment and readable medium - Google Patents
Commodity identification method and device, electronic equipment and readable medium Download PDFInfo
- Publication number
- CN111325049A CN111325049A CN201811524943.9A CN201811524943A CN111325049A CN 111325049 A CN111325049 A CN 111325049A CN 201811524943 A CN201811524943 A CN 201811524943A CN 111325049 A CN111325049 A CN 111325049A
- Authority
- CN
- China
- Prior art keywords
- dimensional
- commodity
- information
- condition
- matching algorithm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 69
- 238000004590 computer program Methods 0.000 claims description 3
- 238000010586 diagram Methods 0.000 description 11
- 230000008569 process Effects 0.000 description 11
- 238000012545 processing Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 5
- 230000007306 turnover Effects 0.000 description 5
- 230000003287 optical effect Effects 0.000 description 4
- 230000006870 function Effects 0.000 description 3
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000003252 repetitive effect Effects 0.000 description 2
- 238000012795 verification Methods 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 239000012636 effector Substances 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000005111 flow chemistry technique Methods 0.000 description 1
- 239000000463 material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000004806 packaging method and process Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 238000012549 training Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/60—Type of objects
- G06V20/64—Three-dimensional objects
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1602—Programme controls characterised by the control system, structure, architecture
- B25J9/161—Hardware, e.g. neural networks, fuzzy logic, interfaces, processor
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1669—Programme controls characterised by programming, planning systems for manipulators characterised by special application, e.g. multi-arm co-operation, assembly, grasping
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
Abstract
The present disclosure provides a method, an apparatus, an electronic device and a readable medium for identifying a commodity, wherein the method comprises: acquiring two-dimensional characteristic information and three-dimensional point cloud information of a commodity; when the two-dimensional characteristic information is judged to meet a first condition, identifying the commodity by using a two-dimensional matching algorithm; when the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition, identifying the commodity by using a three-dimensional matching algorithm; and when the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition, identifying the commodity by using a frame matching algorithm. The commodity identification method, the commodity identification device, the electronic equipment and the readable medium can adaptively match the identification algorithm and improve the accuracy of commodity identification.
Description
Technical Field
The present disclosure relates to the field of warehouse logistics technologies, and in particular, to a method and an apparatus for identifying a commodity, an electronic device, and a computer readable medium.
Background
Under the environment that online shopping is mature, the e-commerce is required to be equipped with a good goods distribution process. In the distribution process, the goods need to be automatically distributed according to the order of the customer so as to package the order. Generally, an automatic goods distribution process is realized by automatically identifying and grabbing goods through a machine, and the automatic grabbing process of the machine may be, for example, a picking process, as shown in fig. 1, where picking by the robot in the container means that the robot takes out a corresponding number of goods from a turnover container as required and puts the goods into a scene at a specified position according to a picking task issued by a system on the basis of visual assistance. The main technical difficulty of the scene is how to accurately identify the central position and the posture of the commodity in the machine vision, so that the robot end effector can accurately grab the commodity.
In identifying the center position and the posture of the commodity, the prior art generally identifies and calculates the commodity through a feature matching algorithm. For densely arranged commodities, gaps among objects cannot be effectively identified, so that matching accuracy is reduced. In addition, most of the existing commodities are square packaging boxes, such as 3C commodities, and are packaged by plastic paper, so that the commodities are weak in texture and high in light reflection, and great difficulty is brought to accurately obtain characteristics; and the existing feature matching algorithm is single, and the matching algorithm suitable for the current scene cannot be intelligently switched according to the characteristics of the recognition object, so that the commodity recognition is low.
In the prior art, a method for identifying commodities by using a deep learning algorithm is provided, however, the method needs a large amount of labeled data to train a network, and the periods of the labeled data and the training network are long, which causes that the method is not suitable for a scene that commodities are changed quickly.
Therefore, a new method, apparatus, electronic device and computer readable medium for identifying an article are needed.
The above information disclosed in this background section is only for enhancement of understanding of the background of the disclosure and therefore it may contain information that does not constitute prior art that is already known to a person of ordinary skill in the art.
Disclosure of Invention
In view of this, the present disclosure provides a method and an apparatus for identifying a commodity, an electronic device, and a computer readable medium, which can adaptively match an identification algorithm and improve the accuracy of commodity identification.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the disclosure.
According to a first aspect of the embodiments of the present disclosure, a method for identifying a commodity is provided, the method including: acquiring two-dimensional characteristic information and three-dimensional point cloud information of a commodity; when the two-dimensional characteristic information is judged to meet a first condition, identifying the commodity by using a two-dimensional matching algorithm; when the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition, identifying the commodity by using a three-dimensional matching algorithm; and when the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition, identifying the commodity by using a frame matching algorithm.
In an exemplary embodiment of the present disclosure, acquiring two-dimensional feature information of a commodity includes: acquiring two-dimensional template information of the commodity; and extracting the commodity according to the two-dimensional template information to generate the two-dimensional feature information.
In an exemplary embodiment of the present disclosure, identifying the commodity using a two-dimensional matching algorithm when it is determined that the two-dimensional feature information satisfies a first condition includes: when the two-dimensional characteristic information is judged to meet a characteristic point threshold condition, confirming that the two-dimensional characteristic information meets a first condition; and identifying the commodity according to the two-dimensional characteristic information and a two-dimensional matching algorithm.
In an exemplary embodiment of the present disclosure, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy a second condition, identifying the commodity using a three-dimensional matching algorithm includes: acquiring three-dimensional template information of the commodity; when the two-dimensional characteristic information does not meet a first condition, judging whether the commodity meets a placing condition or not according to the three-dimensional point cloud information; when the commodity is judged not to meet the placing condition, confirming that the two-dimensional feature information and the three-dimensional point cloud information meet a second condition; and identifying the commodity according to the three-dimensional template information and a three-dimensional matching algorithm.
In an exemplary embodiment of the present disclosure, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy a third condition, identifying the commodity using a frame matching algorithm includes: when the two-dimensional characteristic information does not meet a first condition and the commodity is judged to meet a placing condition according to the three-dimensional point cloud information, confirming that the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition; and identifying the commodity according to the three-dimensional template information and a frame matching algorithm.
In an exemplary embodiment of the present disclosure, further comprising: and when the commodity identification fails by using a two-dimensional matching algorithm, judging whether the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition.
In an exemplary embodiment of the present disclosure, further comprising: and when the commodity identification fails by using the frame matching algorithm, identifying the commodity by using a three-dimensional matching algorithm.
In an exemplary embodiment of the present disclosure, further comprising: and when the commodity identification fails, sending a commodity absence signal.
In an exemplary embodiment of the present disclosure, further comprising: after the commodity is successfully identified, calculating the central position and the posture of the commodity; and sending the central position and the gesture to a picking robot to pick the commodity.
According to a second aspect of the embodiments of the present disclosure, there is provided an article identification device, including: the commodity information module is used for acquiring two-dimensional characteristic information and three-dimensional point cloud information of a commodity; the first identification module is used for identifying the commodity by using a two-dimensional matching algorithm when the two-dimensional characteristic information is judged to meet a first condition; the second identification module is used for identifying the commodity by using a three-dimensional matching algorithm when the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition; and the third identification module is used for identifying the commodity by using a frame matching algorithm when the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition.
According to a third aspect of the embodiments of the present disclosure, an electronic device is provided, which includes: one or more processors; storage means for storing one or more programs; when executed by the one or more processors, cause the one or more processors to implement the article identification method of any of the above.
According to a fourth aspect of the embodiments of the present disclosure, a computer-readable medium is provided, on which a computer program is stored, wherein the program, when executed by a processor, implements the article identification method according to any one of the above.
According to the commodity identification method, the commodity identification device, the electronic equipment and the computer readable medium, the identification algorithm can be adaptively matched, and the accuracy of commodity identification is improved.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. The drawings described below are merely some embodiments of the present disclosure, and other drawings may be derived from those drawings by those of ordinary skill in the art without inventive effort.
FIG. 1 is a schematic diagram illustrating a merchandise picking scenario according to the prior art.
Fig. 2 is a system block diagram illustrating a method and apparatus for article identification according to an example embodiment.
FIG. 3 is a flow chart illustrating a method of article identification according to an exemplary embodiment.
Fig. 4 is a schematic diagram illustrating placement of articles in a turnover box according to an exemplary embodiment.
FIG. 5 is a flow chart illustrating a method of article identification according to an exemplary embodiment.
Fig. 6 is a flowchart illustrating a method of article identification according to another exemplary embodiment.
Fig. 7 is a block diagram illustrating an article identification device according to an exemplary embodiment.
FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.
FIG. 9 is a schematic diagram illustrating a computer-readable storage medium according to an example embodiment.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention may be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known methods, devices, implementations or operations have not been shown or described in detail to avoid obscuring aspects of the invention.
The drawings are merely schematic illustrations of the present invention, in which the same reference numerals denote the same or similar parts, and thus, a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the contents and steps, nor do they necessarily have to be performed in the order described. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation.
The following detailed description of exemplary embodiments of the invention refers to the accompanying drawings.
Fig. 2 is a system block diagram illustrating a method and apparatus for article identification according to an example embodiment.
The server 205 may be a server that provides various services, such as a backend management server (for example only) that provides support for a product identification system operated by a user using the terminal devices 201, 202, 203. The backend management server may analyze and otherwise process data such as the received product identification request, and feed back a processing result (for example, a center position and a posture — just an example) to the terminal device.
The server 205 may, for example, obtain two-dimensional feature information and three-dimensional point cloud information of a commodity; the server 205 may identify the commodity using a two-dimensional matching algorithm, for example, when it is determined that the two-dimensional feature information satisfies the first condition; the server 205 may identify the commodity using a three-dimensional matching algorithm, for example, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition. The server 205 may identify the commodity using a frame matching algorithm, for example, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the third condition.
The server 205 may be a server of one entity, and may also be composed of a plurality of servers, for example, a part of the server 205 may be, for example, used as a product identification task submitting system in the present disclosure, and is configured to obtain a task to be executed with a product identification command; and a part of the server 205 may also be used, for example, as a product identification system in the present disclosure, for obtaining two-dimensional feature information and three-dimensional point cloud information of a product; when the two-dimensional characteristic information is judged to meet the first condition, identifying the commodity by using a two-dimensional matching algorithm; when the two-dimensional characteristic information and the three-dimensional point cloud information meet the second condition, identifying the commodity by using a three-dimensional matching algorithm; and when the two-dimensional characteristic information and the three-dimensional point cloud information meet the third condition, identifying the commodity by using a frame matching algorithm.
It should be noted that the method for product identification provided by the embodiment of the present disclosure may be executed by the server 205, and accordingly, a device for product identification may be disposed in the server 105. And the request end provided for the user to submit the goods identification task and obtain the goods identification result is generally located in the terminal equipment 201, 202, 203.
According to the commodity identification method and the commodity identification device, the identification algorithm can be adaptively matched, and the accuracy of commodity identification is improved.
FIG. 3 is a flow chart illustrating a method of article identification according to an exemplary embodiment. The article identification method 30 includes at least steps S302 to S308.
As shown in fig. 3, in S302, two-dimensional feature information and three-dimensional point cloud information of the product are acquired. The two-dimensional characteristic information can be obtained by extracting from an image shot by a two-dimensional camera. The two-dimensional feature information is image feature information representing commodity feature information. The three-dimensional point cloud information may be acquired by a three-dimensional camera, which may be, for example, but not limited to, a binocular camera.
In one embodiment, acquiring two-dimensional characteristic information of the commodity comprises: acquiring two-dimensional template information of a commodity; and extracting the commodity according to the two-dimensional template information to generate two-dimensional characteristic information. The two-dimensional template information is template information of a single commodity, for example, but not limited to, image information of each side of the commodity, and specifically, may be feature information of a point, a line, and a side of each side. When the commodity is extracted, the commodity image can be extracted according to two-dimensional template information to obtain two-dimensional characteristic information of the commodity. The two-dimensional feature information may be, for example, but not limited to, color features, texture features, shape features, and spatial location features, among others.
In S304, when it is determined that the two-dimensional feature information satisfies the first condition, the commodity is identified using a two-dimensional matching algorithm. The first condition is used for judging whether the two-dimensional matching algorithm is suitable for commodity identification of the current scene. For example, the first condition may be used to determine whether the feature point number of the two-dimensional feature information reaches a threshold condition, and when the threshold condition is reached, it is considered that a good recognition effect can be obtained when the two-dimensional feature information is used for product recognition. The two-dimensional matching algorithm is an algorithm for matching the commodities according to the two-dimensional characteristic information and the two-dimensional template information to identify the commodities. When the two-dimensional matching algorithm is suitable for commodity identification of the current scene, the identification accuracy rate of the two-dimensional matching algorithm is higher than that of other matching algorithms.
In one embodiment, when the two-dimensional feature information is judged to meet the feature point threshold condition, the two-dimensional feature information is confirmed to meet a first condition; and identifying the commodity according to the two-dimensional characteristic information and a two-dimensional matching algorithm. And when the number of the feature points in the two-dimensional feature information is greater than the feature point threshold value, the two-dimensional feature information is considered to meet the feature point threshold value condition. The specific numerical value of the feature point threshold is obtained from experience or test, and is the minimum feature point number for which the recognition result is effective when the two-dimensional matching algorithm is used for recognizing the commodity.
In one embodiment, further comprising: and when the commodity identification by using the two-dimensional matching algorithm fails, judging whether the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition. The case of failed product identification is the case of 0 number of matched products, and when the matching fails or the products do not exist, the identification fails. Under the condition, the condition judgment is carried out on the two-dimensional characteristic information and the three-dimensional point cloud information again, and the commodity is identified again by using the three-dimensional matching algorithm or the frame matching algorithm according to the condition judgment result so as to improve the identification accuracy, wherein the condition judgment process is shown in S306 and S308. The frame matching algorithm is an algorithm which only depends on the size information of the commodity for matching, does not depend on the feature information of the feature template, and only depends on the external size of the matched object. In the frame matching algorithm, the purpose of identifying the object is achieved by matching lines with the same size as the template in the two-dimensional image. The frame matching algorithm does not depend on the texture features of the outer surface of the template, and in comparison with algorithms which depend on feature information, such as a two-dimensional matching algorithm, a three-dimensional matching algorithm and the like, the frame matching algorithm depends on less prior knowledge, so that the frame matching algorithm has a better recognition effect on the recognition of objects with poor texture or easily-reflective materials.
In S306, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition, the product is identified using a three-dimensional matching algorithm. Wherein the second condition is used for judging that the current scene is not suitable for the two-dimensional matching algorithm but is suitable for the three-dimensional matching algorithm. For example, when the two-dimensional feature information does not satisfy the first condition and the three-dimensional point cloud information is judged to satisfy the condition of using the three-dimensional matching algorithm, it is determined that the current two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition. The three-dimensional matching algorithm is to use three-dimensional template information of the commodity to detect a target object in the three-dimensional point cloud information according to the similarity between the three-dimensional point cloud information and the template.
In one embodiment, three-dimensional template information of a commodity is acquired; when the two-dimensional characteristic information does not meet the first condition, judging whether the commodity meets the placing condition or not according to the three-dimensional point cloud information; when the commodity is judged to meet the non-placement condition, confirming that the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition; and identifying the commodity according to the three-dimensional template information and a three-dimensional matching algorithm. And the placing condition is whether the goods are placed densely, namely when the three-dimensional point cloud information is analyzed to confirm that the goods are not placed densely, namely the goods are placed in a scattered manner, the current scene is considered not to meet the placing condition. When the commodities are identified, three-dimensional matching is carried out in the three-dimensional point cloud information according to the three-dimensional template information so as to identify the commodities.
In one embodiment, further comprising: when the commodity identification fails, a commodity absence signal is transmitted. The case that the commodity identification fails is the case that the number of the matched commodities is 0, and the commodity absence signal is sent to indicate that the current turnover box is empty. Further, the conveyor belt can be excited to operate according to the commodity absence signal so as to identify the commodities in the next turnover box.
In S308, when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the third condition, the commodity is identified using a frame matching algorithm. The third condition is used for judging that the current scene is not suitable for the two-dimensional matching algorithm, and is more suitable for the frame matching algorithm compared with the three-dimensional matching algorithm. For example, when the two-dimensional feature information does not satisfy the first condition and the three-dimensional point cloud information is determined to satisfy the placement condition, it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the third condition. The frame matching algorithm only depends on the size information of the commodity, namely three-dimensional template information. For goods without textures or weak textures, compared with a two-dimensional matching algorithm and a three-dimensional matching algorithm, the algorithm has better identification accuracy and depends on less prior information. The frame matching algorithm has a better recognition effect on picking operation in the square commodity box.
In one embodiment, when the two-dimensional characteristic information does not meet the first condition and the commodity is judged to not meet the placing condition according to the three-dimensional point cloud information, the two-dimensional characteristic information and the three-dimensional point cloud information are confirmed to meet a third condition; and identifying the commodity according to the three-dimensional template information and a frame matching algorithm. The placing condition is whether the placing is scattered or not, namely the placing condition is not satisfied when the three-dimensional point cloud information is analyzed to confirm that the placing is not scattered. The schematic placing diagram of the square commodity when the placing condition is not met is shown in fig. 4. When the commodity is identified, the frame of the commodity is identified in the two-dimensional characteristic information according to the three-dimensional template information so as to identify the commodity.
In one embodiment, further comprising: and when the commodity identification by using the frame matching algorithm fails, identifying the commodity by using the three-dimensional matching algorithm. When the turnover box is empty or the frame matching fails, the commodity identification fails by using the frame matching algorithm.
In one embodiment, further comprising: after the commodity is successfully identified, calculating the central position and the posture of the commodity; and sending the central position and the gesture to a picking robot to pick the goods. The center position and the posture of each commodity can be calculated according to the recognition result, so that the robot can pick the commodities according to the center position and the posture.
According to the commodity identification method, the two-dimensional characteristic information and the three-dimensional point cloud information are analyzed and judged, and a commodity identification algorithm suitable for the current scene is selected according to the judgment result. The commodity identification method can adaptively match the identification algorithm and improve the accuracy of commodity identification.
FIG. 5 is a flow chart illustrating a method of article identification according to an exemplary embodiment. The article identification method 50 includes at least steps S502 to S508.
As shown in fig. 5, in S502, three-dimensional template information of the product is acquired. The three-dimensional template information may be, for example, but not limited to, size information of the product.
In S504, when the two-dimensional feature information does not satisfy the first condition, it is determined whether the commodity satisfies the placement condition according to the three-dimensional point cloud information. The first condition is used for judging whether the current scene is suitable for the two-dimensional matching algorithm. The placing condition is whether the commodities are placed densely or not.
In S506, when it is determined that the commodity does not satisfy the placement condition, it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition. When the commodity is judged to meet the placing condition, analysis can be carried out according to the three-dimensional point cloud information so as to confirm whether the commodity is placed densely.
In S508, the product is identified according to the three-dimensional template information and the three-dimensional matching algorithm. And matching the three-dimensional point cloud information by using a three-dimensional matching algorithm according to the three-dimensional template information so as to identify the commodity.
According to the commodity identification method, the commodity is identified by using the frame matching algorithm when the third condition is met, and the identification efficiency and the accuracy can be improved when the regularly placed square commodity is identified.
According to the commodity identification method, the commodities are identified by adopting different identification algorithms according to different scenes, and the identification accuracy of the commodities can be improved.
It should be clearly understood that this disclosure describes how to make and use particular examples, but the principles of this disclosure are not limited to any details of these examples. Rather, these principles can be applied to many other embodiments based on the teachings of the present disclosure.
Fig. 6 is a flowchart illustrating a method of article identification according to another exemplary embodiment. As shown in fig. 6, first, two-dimensional template information (for example, a two-dimensional image of six surfaces of a commodity) and three-dimensional template information (for example, commodity size information: length, width, and height) of the commodity are collected, and the two-dimensional template information and the three-dimensional template information may be collected collectively, for example, when the commodity is put in storage.
When the commodity identification method is realized, firstly, two-dimensional characteristic information is extracted from a two-dimensional image according to two-dimensional template information of a commodity, and if the number of characteristic points in the two-dimensional characteristic information exceeds a set characteristic point threshold value, a target object to be captured is identified by matching the two-dimensional template information on a scene two-dimensional image, as shown in ① in a flow chart.
In this case, the distribution condition of the three-dimensional point cloud information of the scene is analyzed to judge whether the commodity placement scene is randomly placed or densely placed, and if the commodity placement scene is randomly placed, the commodity to be picked is directly identified on the three-dimensional point cloud information of the scene by using the three-dimensional template information of the commodity, as shown by ③ in the flow chart.
If the number of the feature points of the two-dimensional template information does not reach the threshold value and the commodities are judged to be densely arranged according to the distribution of the three-dimensional point cloud information of the scene, identifying the commodities to be picked by adopting a frame matching algorithm, as shown by ② in the flow chart, and if the commodities cannot be identified by frame matching, identifying the commodities to be picked again on the three-dimensional point cloud of the scene, as shown by ③ in the flow chart.
If the three algorithms do not identify the commodities, the empty container is judged, and the commodities to be sorted do not exist. If the commodity is identified according to the three algorithms, the central position and the posture of the commodity are calculated and sent to the robot to be sorted.
According to the commodity identification method, the two-dimensional characteristic information and the three-dimensional point cloud information are analyzed and judged, and a commodity identification algorithm suitable for the current scene is selected according to the judgment result. The commodity identification method can adaptively match the identification algorithm and improve the accuracy of commodity identification.
According to the commodity identification method, when the two-dimensional characteristic information does not meet the threshold condition and the commodities are densely arranged, the commodities are identified by using the frame matching algorithm, and the identification efficiency and the accuracy can be improved when the square commodities which are placed orderly are identified.
Those skilled in the art will appreciate that all or part of the steps implementing the above embodiments are implemented as computer programs executed by a CPU. When executed by the CPU, performs the functions defined by the above-described methods provided by the present disclosure. The program may be stored in a computer readable storage medium, which may be a read-only memory, a magnetic or optical disk, or the like.
Furthermore, it should be noted that the above-mentioned figures are only schematic illustrations of the processes involved in the methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
The following are embodiments of the disclosed apparatus that may be used to perform embodiments of the disclosed methods. For details not disclosed in the embodiments of the apparatus of the present disclosure, refer to the embodiments of the method of the present disclosure.
Fig. 7 is a block diagram illustrating an article identification device according to an exemplary embodiment. Referring to fig. 7, the article recognition device 70 includes at least: an item information module 702, a first identification module 704, a second identification module 706, and a third identification module 708.
In the commodity identification device, a commodity information module 702 is used for acquiring two-dimensional feature information and three-dimensional point cloud information of a commodity. The two-dimensional characteristic information can be obtained by extracting from an image shot by a two-dimensional camera. The two-dimensional feature information is image feature information representing commodity feature information. The three-dimensional point cloud information may be acquired by a three-dimensional camera, which may be, for example, but not limited to, a binocular camera.
In one embodiment, the merchandise information module 702 is configured to obtain two-dimensional template information of merchandise; and extracting the commodity according to the two-dimensional template information to generate two-dimensional characteristic information.
The first identification module 704 is configured to identify the commodity by using a two-dimensional matching algorithm when it is determined that the two-dimensional feature information satisfies a first condition. The first condition is used for judging whether the two-dimensional matching algorithm is suitable for commodity identification of the current scene. For example, the first condition may be used to determine whether the feature point number of the two-dimensional feature information reaches a threshold condition, and when the threshold condition is reached, it is considered that a good recognition effect can be obtained when the two-dimensional feature information is used for product recognition. The two-dimensional matching algorithm is an algorithm for matching the commodities according to the two-dimensional characteristic information and the two-dimensional template information to identify the commodities. When the two-dimensional matching algorithm is suitable for commodity identification of the current scene, the identification accuracy rate of the two-dimensional matching algorithm is higher than that of other matching algorithms.
In one embodiment, the first identification module 704 is configured to confirm that the two-dimensional feature information satisfies the first condition when the two-dimensional feature information is determined to satisfy the feature point threshold condition; and identifying the commodity according to the two-dimensional characteristic information and a two-dimensional matching algorithm.
In one embodiment, the first identification module 704 is configured to determine whether the two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition when the two-dimensional matching algorithm fails to identify the commodity.
The second identification module 706 is configured to identify the commodity by using a three-dimensional matching algorithm when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition. Wherein the second condition is used for judging that the current scene is not suitable for the two-dimensional matching algorithm but is suitable for the three-dimensional matching algorithm. For example, when the two-dimensional feature information does not satisfy the first condition and the three-dimensional point cloud information is judged to satisfy the condition of using the three-dimensional matching algorithm, it is determined that the current two-dimensional feature information and the three-dimensional point cloud information satisfy the second condition. The three-dimensional matching algorithm is to use three-dimensional template information of the commodity to detect a target object in the three-dimensional point cloud information according to the similarity between the three-dimensional point cloud information and the template.
In one embodiment, the second identification module 706 is configured to obtain three-dimensional template information of the product; when the two-dimensional characteristic information does not meet the first condition, judging whether the commodity meets the placing condition or not according to the three-dimensional point cloud information; when the commodity is judged not to meet the placing condition, the two-dimensional characteristic information and the three-dimensional point cloud information are confirmed to meet a second condition; and identifying the commodity according to the three-dimensional template information and a three-dimensional matching algorithm.
In one embodiment, the second identification module 706 is further configured to send an article absence signal when the article identification fails.
The third identifying module 708 is configured to identify the commodity by using a frame matching algorithm when it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy a third condition.
The third condition is used for judging that the current scene is not suitable for the two-dimensional matching algorithm, and is more suitable for the frame matching algorithm compared with the three-dimensional matching algorithm. For example, when the two-dimensional feature information does not satisfy the first condition and the three-dimensional point cloud information is determined to satisfy the placement condition, it is determined that the two-dimensional feature information and the three-dimensional point cloud information satisfy the third condition. The frame matching algorithm only depends on the size information of the commodity, namely three-dimensional template information. For goods without textures or weak textures, compared with a two-dimensional matching algorithm and a three-dimensional matching algorithm, the algorithm has better identification accuracy and depends on less prior information. The frame matching algorithm has a better recognition effect on picking operation in the square commodity box.
In one embodiment, the third identifying module 708 is configured to confirm that the two-dimensional feature information and the three-dimensional point cloud information satisfy the third condition when the two-dimensional feature information does not satisfy the first condition and the commodity is determined to satisfy the placing condition according to the three-dimensional point cloud information; and identifying the commodity according to the three-dimensional template information and a frame matching algorithm.
In one embodiment, the third identification module 708 is further configured to identify the product using a three-dimensional matching algorithm when the product fails to be identified using the border matching algorithm.
In one embodiment, further comprising: after the commodity is successfully identified, calculating the central position and the posture of the commodity; and sending the central position and the gesture to a picking robot to pick the goods.
According to the commodity identification device disclosed by the invention, the two-dimensional characteristic information and the three-dimensional point cloud information are analyzed and judged, and a commodity identification algorithm suitable for the current scene is selected according to the judgment result. The commodity identification method can adaptively match the identification algorithm and improve the accuracy of commodity identification.
FIG. 8 is a block diagram illustrating an electronic device in accordance with an example embodiment.
An electronic device 200 according to this embodiment of the present disclosure is described below with reference to fig. 8. The electronic device 200 shown in fig. 8 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 8, the electronic device 200 is embodied in the form of a general purpose computing device. The components of the electronic device 200 may include, but are not limited to: at least one processing unit 210, at least one memory unit 220, a bus 230 connecting different system components (including the memory unit 220 and the processing unit 210), a display unit 240, and the like.
Wherein the storage unit stores program code executable by the processing unit 210 to cause the processing unit 210 to perform the steps according to various exemplary embodiments of the present disclosure described in the above-mentioned electronic prescription flow processing method section of the present specification. For example, the processing unit 210 may perform the steps as shown in fig. 3, 5, 6.
The memory unit 220 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)2201 and/or a cache memory unit 2202, and may further include a read only memory unit (ROM) 2203.
The storage unit 220 may also include a program/utility 2204 having a set (at least one) of program modules 2205, such program modules 2205 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 200 may also communicate with one or more external devices 300 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 200, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 200 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 250. Also, the electronic device 200 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 260. The network adapter 260 may communicate with other modules of the electronic device 200 via the bus 230. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 200, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the embodiments of the present disclosure.
Fig. 9 schematically illustrates a computer-readable storage medium in an exemplary embodiment of the disclosure.
Referring to fig. 9, a program product 400 for implementing the above method according to an embodiment of the present disclosure is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in this document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The computer readable storage medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable storage medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a readable storage medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
The computer readable medium carries one or more programs which, when executed by a device, cause the computer readable medium to perform the functions of: acquiring verification information on an article to be received through a scanning device of the intelligent express box; when the verification information meets a preset condition, generating a cabin door opening signal and a camera opening signal of the intelligent express box; acquiring a cabin door closing signal according to a first user operation; and generating a video recording closing signal of the intelligent express box according to the cabin door closing signal.
Those skilled in the art will appreciate that the modules described above may be distributed in the apparatus according to the description of the embodiments, or may be modified accordingly in one or more apparatuses unique from the embodiments. The modules of the above embodiments may be combined into one module, or further split into multiple sub-modules.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Exemplary embodiments of the present disclosure are specifically illustrated and described above. It is to be understood that the present disclosure is not limited to the precise arrangements, instrumentalities, or instrumentalities described herein; on the contrary, the disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.
Claims (12)
1. A method for identifying an article, comprising:
acquiring two-dimensional characteristic information and three-dimensional point cloud information of a commodity;
when the two-dimensional characteristic information is judged to meet a first condition, identifying the commodity by using a two-dimensional matching algorithm;
when the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition, identifying the commodity by using a three-dimensional matching algorithm; and
and when the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition, identifying the commodity by using a frame matching algorithm.
2. The method of claim 1, wherein obtaining two-dimensional feature information of the commodity comprises:
acquiring two-dimensional template information of the commodity; and
and extracting the commodity according to the two-dimensional template information to generate the two-dimensional characteristic information.
3. The method of claim 2, wherein identifying the good using a two-dimensional matching algorithm upon determining that the two-dimensional feature information satisfies a first condition comprises:
when the two-dimensional characteristic information is judged to meet a characteristic point threshold condition, confirming that the two-dimensional characteristic information meets a first condition; and
and identifying the commodity according to the two-dimensional characteristic information and a two-dimensional matching algorithm.
4. The method of claim 1, wherein identifying the good using a three-dimensional matching algorithm upon determining that the two-dimensional feature information and the three-dimensional point cloud information satisfy a second condition comprises:
acquiring three-dimensional template information of the commodity;
when the two-dimensional characteristic information does not meet a first condition, judging whether the commodity meets a placing condition or not according to the three-dimensional point cloud information;
when the commodity is judged not to meet the placing condition, confirming that the two-dimensional feature information and the three-dimensional point cloud information meet a second condition; and
and identifying the commodity according to the three-dimensional template information and a three-dimensional matching algorithm.
5. The method of claim 1, wherein identifying the good using a bounding box matching algorithm when the two-dimensional feature information and the three-dimensional point cloud information are determined to satisfy a third condition comprises:
when the two-dimensional characteristic information does not meet a first condition and the commodity is judged to meet a placing condition according to the three-dimensional point cloud information, confirming that the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition; and
and identifying the commodity according to the three-dimensional template information and a frame matching algorithm.
6. The method of claim 1, further comprising:
and when the commodity identification fails by using a two-dimensional matching algorithm, judging whether the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition.
7. The method of claim 1, further comprising:
and when the commodity identification fails by using the frame matching algorithm, identifying the commodity by using a three-dimensional matching algorithm.
8. The method of claim 1, further comprising:
and when the commodity identification fails, sending a commodity absence signal.
9. The method of claim 1, further comprising:
after the commodity is successfully identified, calculating the central position and the posture of the commodity; and
and sending the central position and the gesture to a picking robot to pick the commodity.
10. An article identification device, comprising:
the commodity information module is used for acquiring two-dimensional characteristic information and three-dimensional point cloud information of a commodity;
the first identification module is used for identifying the commodity by using a two-dimensional matching algorithm when the two-dimensional characteristic information is judged to meet a first condition;
the second identification module is used for identifying the commodity by using a three-dimensional matching algorithm when the two-dimensional characteristic information and the three-dimensional point cloud information meet a second condition; and
and the third identification module is used for identifying the commodity by using a frame matching algorithm when the two-dimensional characteristic information and the three-dimensional point cloud information meet a third condition.
11. An electronic device, comprising:
one or more processors;
storage means for storing one or more programs;
when executed by the one or more processors, cause the one or more processors to implement the method of any one of claims 1-9.
12. A computer-readable medium, on which a computer program is stored, which, when being executed by a processor, carries out the method according to any one of claims 1-9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811524943.9A CN111325049A (en) | 2018-12-13 | 2018-12-13 | Commodity identification method and device, electronic equipment and readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811524943.9A CN111325049A (en) | 2018-12-13 | 2018-12-13 | Commodity identification method and device, electronic equipment and readable medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111325049A true CN111325049A (en) | 2020-06-23 |
Family
ID=71172313
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811524943.9A Pending CN111325049A (en) | 2018-12-13 | 2018-12-13 | Commodity identification method and device, electronic equipment and readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111325049A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011257811A (en) * | 2010-06-04 | 2011-12-22 | Shinshu Univ | Merchandise search system, and method for searching merchandise in merchandise search system |
| WO2014191055A1 (en) * | 2013-05-31 | 2014-12-04 | Longsand Limited | Three-dimensional object modeling |
| CN104680152A (en) * | 2015-03-12 | 2015-06-03 | 睿寻科技（北京）有限责任公司 | Cylindrical trademark recognition method and device |
| JP2016126515A (en) * | 2014-12-26 | 2016-07-11 | 京セラオプテック株式会社 | Object recognition scanner system |
| CN108551658A (en) * | 2017-12-18 | 2018-09-18 | 上海云拿智能科技有限公司 | Object positioning system and localization method |
| CN108734058A (en) * | 2017-04-18 | 2018-11-02 | 百度在线网络技术（北京）有限公司 | Obstacle identity recognition methods, device, equipment and storage medium |
| CN108846792A (en) * | 2018-05-23 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Image processing method, device, electronic equipment and computer-readable medium |
-
2018
- 2018-12-13 CN CN201811524943.9A patent/CN111325049A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011257811A (en) * | 2010-06-04 | 2011-12-22 | Shinshu Univ | Merchandise search system, and method for searching merchandise in merchandise search system |
| WO2014191055A1 (en) * | 2013-05-31 | 2014-12-04 | Longsand Limited | Three-dimensional object modeling |
| JP2016126515A (en) * | 2014-12-26 | 2016-07-11 | 京セラオプテック株式会社 | Object recognition scanner system |
| CN104680152A (en) * | 2015-03-12 | 2015-06-03 | 睿寻科技（北京）有限责任公司 | Cylindrical trademark recognition method and device |
| CN108734058A (en) * | 2017-04-18 | 2018-11-02 | 百度在线网络技术（北京）有限公司 | Obstacle identity recognition methods, device, equipment and storage medium |
| CN108551658A (en) * | 2017-12-18 | 2018-09-18 | 上海云拿智能科技有限公司 | Object positioning system and localization method |
| CN108846792A (en) * | 2018-05-23 | 2018-11-20 | 腾讯科技（深圳）有限公司 | Image processing method, device, electronic equipment and computer-readable medium |
Non-Patent Citations (3)
| Title |
|---|
| RICHTSFELD A 等: "Basic object shape detection and tracking using perceptual organization", PROCEEDING OF 2009IEEE INTERNATIONAL CONFERENCE ON ADVANCED ROBOTICS * |
| 王虎;朴红吉;吴振宇;冯林;: "DataMatrix二维条码在票务系统中的应用与研究", 计算机与数字工程, no. 03 * |
| 魏玉锋;梁冬泰;梁丹;邢淑敏;: "基于多模态信息的机器人视觉识别与定位研究", 光电工程, no. 02 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10664692B2 (en) | Visual task feedback for workstations in materials handling facilities | |
| CN110197309B (en) | Order processing method, device, equipment and storage medium | |
| JP7131921B2 (en) | Automated Object Tracking in Video Feeds Using Machine Learning | |
| WO2019090268A1 (en) | Contextual training systems and methods | |
| US11087172B2 (en) | Systems and methods for creating training data | |
| WO2019071930A1 (en) | Method and apparatus for sorting goods | |
| CN109472523A (en) | Method and apparatus for sorting cargo | |
| WO2023020105A1 (en) | Supplies inventory method and apparatus, and device and storage medium | |
| Tonioni et al. | Product recognition in store shelves as a sub-graph isomorphism problem | |
| CN110930221B (en) | Abnormal order processing method, storage medium and computer equipment | |
| US20180074162A1 (en) | System and Methods for Identifying an Action Based on Sound Detection | |
| CN110555356A (en) | Self-checkout system, method and device | |
| WO2019080674A1 (en) | Self-service checkout device, method, apparatus, medium and electronic device | |
| CN109733783A (en) | A kind of cargo restocking method, apparatus, electronic equipment and storage medium | |
| CN110516628A (en) | Shelf vacant locations merchandise news acquisition methods, system, equipment and storage medium | |
| WO2023015916A1 (en) | Three-dimensional sorting method, and three-dimensional sorting robot and system | |
| CN111666816A (en) | Method, device and equipment for detecting state of logistics piece | |
| US10372753B2 (en) | System for verifying physical object absences from assigned regions using video analytics | |
| CN108446693B (en) | Marking method, system, equipment and storage medium of target to be identified | |
| CN111325049A (en) | Commodity identification method and device, electronic equipment and readable medium | |
| US20220203547A1 (en) | System and method for improving automated robotic picking via pick planning and interventional assistance | |
| CN111753614A (en) | Commodity shelf monitoring method and device | |
| WO2023015917A1 (en) | Three-dimensional sorting robot-based data processing method and three-dimensional sorting robot | |
| CN114170254A (en) | Image distribution recognition method, system, device and storage medium | |
| CN113978987A (en) | Pallet object packaging and picking method, device, equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| TA01 | Transfer of patent application right |
Effective date of registration: 20210305 Address after: 101, 1st floor, building 2, yard 20, Suzhou street, Haidian District, Beijing 100080 Applicant after: Beijing Jingbangda Trading Co.,Ltd. Address before: 100086 8th Floor, 76 Zhichun Road, Haidian District, Beijing Applicant before: BEIJING JINGDONG SHANGKE INFORMATION TECHNOLOGY Co.,Ltd. Applicant before: BEIJING JINGDONG CENTURY TRADING Co.,Ltd. Effective date of registration: 20210305 Address after: Room a1905, 19 / F, building 2, No. 18, Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Beijing Jingdong Qianshi Technology Co.,Ltd. Address before: 101, 1st floor, building 2, yard 20, Suzhou street, Haidian District, Beijing 100080 Applicant before: Beijing Jingbangda Trading Co.,Ltd. |
|
| TA01 | Transfer of patent application right |