CN113722203A - Program testing method and device, electronic device and computer readable storage medium - Google Patents
Program testing method and device, electronic device and computer readable storage medium Download PDFInfo
- Publication number
- CN113722203A CN113722203A CN202010456655.5A CN202010456655A CN113722203A CN 113722203 A CN113722203 A CN 113722203A CN 202010456655 A CN202010456655 A CN 202010456655A CN 113722203 A CN113722203 A CN 113722203A
- Authority
- CN
- China
- Prior art keywords
- image
- test
- similarity
- block
- case
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3684—Test management for test design, e.g. generating new test cases
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/36—Preventing errors by testing or debugging software
- G06F11/3668—Software testing
- G06F11/3672—Test management
- G06F11/3688—Test management for test execution, e.g. scheduling of test suites
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Quality & Reliability (AREA)
- Computer Hardware Design (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Evolutionary Computation (AREA)
- Evolutionary Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Debugging And Monitoring (AREA)
Abstract
The disclosure relates to a program testing method and device, electronic equipment and a computer readable storage medium, relates to the technical field of software testing, and can be applied to a front-end automatic testing scene. The program testing method comprises the following steps: determining and executing a test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image; determining the similarity between the reference image and the test image as a first similarity, and judging whether the first similarity is smaller than a first threshold value; if the first similarity is smaller than a first threshold value, respectively carrying out image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image; and determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block. The method and the device can avoid human errors caused by manual participation in the existing front-end testing scheme, save labor cost and improve testing efficiency.
Description
Technical Field
The present disclosure relates to the field of software testing technologies, and in particular, to a program testing method, a program testing apparatus, an electronic device, and a computer-readable storage medium.
Background
Software testing refers to the process of testing a system using manual or automated means to verify that the functionality meets a predetermined requirement. Currently, most front-end automated testing is typically based on a "page operations-element positioning-object assertions" scheme that requires pre-preparation of the expected results of both positioning the element machines and assertion usage into test cases.
The existing front-end automatic test scheme has the following problems: firstly, the preparation workload of the test case is large, the maintenance cost is high, and the test occupies certain labor cost in the whole software development process; secondly, by manually inputting expected results used by assertions, human errors may exist, the software testing quality is affected by the capability and experience of testers, and the testing process is easy to cause inconsistency of the testing process and results due to the human errors. Third, for agile development models, test cases are difficult to adapt to rapidly changing product requirements, taking a lot of time to maintain automated test cases greatly limits the range of use of automated testing.
It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.
Disclosure of Invention
The present disclosure is directed to a program testing method, a program testing apparatus, an electronic device, and a computer-readable storage medium, so as to overcome, at least to a certain extent, the problems that an existing front-end automated testing scheme involves too many manual interventions, which may result in human errors and high occupied human costs.
Additional features and advantages of the disclosure will be set forth in the detailed description which follows, or in part will be obvious from the description, or may be learned by practice of the invention.
According to a first aspect of the present disclosure, there is provided a program testing method including: determining a test case of the application program, and executing the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image; determining the similarity between the reference image and the test image as a first similarity, and judging whether the first similarity is smaller than a first threshold value; if the first similarity is smaller than a first threshold value, respectively carrying out image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image; and determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block.
Optionally, executing the test case to determine a case image corresponding to the test case includes: determining an image storage directory corresponding to each test case; determining a test script corresponding to the test case, and executing the test script to obtain a case image; and respectively storing the case images to the image storage catalogues corresponding to the test cases.
Optionally, determining that the similarity between the reference image and the test image is a first similarity includes: extracting image features of the reference image as reference image features, and extracting image features of the test image as test image features; the reference image features are compared to the test image features to determine a first similarity.
Optionally, the image segmentation processing is performed on the reference image and the test image respectively to obtain a block reference image and a block test image, and the method includes: determining a target number; the target quantity is the quantity of images obtained after image segmentation processing; performing image segmentation processing on the reference image to obtain a target number of block reference images; and carrying out image segmentation processing on the test images to obtain the target number of block test images.
Optionally, determining a target difference block according to the block reference image and the block test image includes: respectively calculating the similarity between each block reference image and each block test image to obtain a plurality of second similarities; and comparing each second similarity with a second threshold value to determine target difference blocks according to the comparison result.
Optionally, comparing each second similarity with a second threshold to determine a target difference block according to the comparison result, including: determining a first image number corresponding to each block reference image, and determining a second image number corresponding to each block test image; generating a plurality of similarity numbers according to the first image number and the second image number; determining second similarity corresponding to each similarity number, and judging whether the second similarity is smaller than a second threshold value; and determining a target similarity number with the second similarity smaller than a second threshold value, and determining the block test image corresponding to the target similarity number as a target difference block.
Optionally, the program testing method further includes: acquiring a target case image corresponding to a target test case; and determining an abnormal code segment of the target test case according to the target case image so as to adjust the abnormal code segment.
According to a second aspect of the present disclosure, there is provided a program testing apparatus comprising: the case image determining module is used for determining a test case of the application program and executing the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image; the similarity comparison module is used for determining the similarity between the reference image and the test image as a first similarity and judging whether the first similarity is smaller than a first threshold value; the image blocking module is used for respectively carrying out image segmentation processing on the reference image and the test image to obtain a blocked reference image and a blocked test image if the first similarity is smaller than a first threshold value; and the difference block verification module is used for determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block.
Optionally, the use case image determining module includes a use case image determining unit, configured to determine an image storage directory corresponding to each test case; determining a test script corresponding to the test case, and executing the test script to obtain a case image; and respectively storing the case images to the image storage catalogues corresponding to the test cases.
Optionally, the similarity comparison module includes a similarity determination unit, configured to extract an image feature of the reference image as a reference image feature, and extract an image feature of the test image as a test image feature; the reference image features are compared to the test image features to determine a first similarity.
Optionally, the image blocking module includes an image blocking unit, configured to determine the target number; the target quantity is the quantity of images obtained after image segmentation processing; performing image segmentation processing on the reference image to obtain a target number of block reference images; and carrying out image segmentation processing on the test images to obtain the target number of block test images.
Optionally, the difference blocking verification module includes a difference blocking determination unit, configured to calculate similarities between each blocking reference image and each blocking test image, respectively, so as to obtain a plurality of second similarities; and comparing each second similarity with a second threshold value to determine target difference blocks according to the comparison result.
Optionally, the difference block determining unit includes a difference block determining subunit, configured to determine first image numbers corresponding to the block reference images, and determine second image numbers corresponding to the block test images; generating a plurality of similarity numbers according to the first image number and the second image number; determining second similarity corresponding to each similarity number, and judging whether the second similarity is smaller than a second threshold value; and determining a target similarity number with the second similarity smaller than a second threshold value, and determining the block test image corresponding to the target similarity number as a target difference block.
Optionally, the program testing apparatus further includes a code segment determining module, configured to obtain a target case image corresponding to the target test case; and determining an abnormal code segment of the target test case according to the target case image so as to adjust the abnormal code segment.
According to a third aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory having computer readable instructions stored thereon which, when executed by the processor, implement a program testing method according to any one of the above.
According to a fourth aspect of the present disclosure, there is provided a computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements a program testing method according to any one of the above.
The technical scheme provided by the disclosure can comprise the following beneficial effects:
the program testing method in the exemplary embodiment of the present disclosure determines a test case of an application program, and executes the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image; determining the similarity between the reference image and the test image as a first similarity, and judging whether the first similarity is smaller than a first threshold value; if the first similarity is smaller than a first threshold value, respectively carrying out image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image; and determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block. According to the program test method, on one hand, the image similarity calculation and image segmentation technology is applied to program test, and the image with the similarity smaller than the threshold is segmented, so that the target abnormal block in the case image can be accurately positioned. On the other hand, the similarity is compared with the threshold, and the position where the difference is generated between the test image and the reference image is positioned according to the comparison result, so that the manual intervention in the program test can be reduced, the labor is saved, and the test efficiency is improved. In another aspect, because manual intervention is reduced in the program testing process, certain manual errors can be avoided, and the testing quality of the program is ensured.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
FIG. 1 schematically illustrates a flow chart of a program testing method according to an exemplary embodiment of the present disclosure;
FIG. 2 schematically illustrates an overall technical flow diagram of a program testing method according to an exemplary embodiment of the present disclosure;
FIG. 3 schematically illustrates a use case diagram before image segmentation for a use case image, according to an exemplary embodiment of the present disclosure;
FIG. 4 schematically illustrates a result graph after image segmentation processing is performed on a use case image according to an exemplary embodiment of the present disclosure;
FIG. 5 schematically illustrates a block diagram of a program testing apparatus according to an exemplary embodiment of the present disclosure;
FIG. 6 schematically illustrates a block diagram of an electronic device according to an exemplary embodiment of the present disclosure;
fig. 7 schematically illustrates a schematic diagram of a computer-readable storage medium according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The same reference numerals denote the same or similar parts in the drawings, and thus, a repetitive description thereof will be omitted.
Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and so forth. In other instances, well-known structures, methods, devices, implementations, materials, or operations are not shown or described in detail to avoid obscuring aspects of the disclosure.
The block diagrams shown in the figures are functional entities only and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in the form of software, or in one or more software-hardened modules, or in different networks and/or processor devices and/or microcontroller devices.
In agile software development, in the face of fast iteration of functions, the human cost occupied by software testing becomes one of the most expensive processes. In recent years, computer vision technology has been successfully applied to different fields, wherein image segmentation and image retrieval technology has been successful to some extent under various use scenes. For the test field, the deep learning technology is not applied at present, and the test process is easy to be inconsistent due to manual errors, so that the test is still carried out and the result is analyzed through manual work, the test efficiency is low, and the software quality can be influenced by the capability and experience of testers. Under the technical background, the image segmentation technology and the image retrieval technology are applied to the field of software automation test to make up for the defects of the existing automation framework technology in program test, and become a research hotspot in the field of software test.
Based on this, in the present exemplary embodiment, first, a program testing method is provided, which may be implemented by using a server, or a terminal device, wherein the terminal described in the present disclosure may include a mobile terminal such as a mobile phone, a tablet computer, a notebook computer, a palm top computer, a Personal Digital Assistant (PDA), and a fixed terminal such as a desktop computer. Fig. 1 schematically illustrates a schematic diagram of a program testing method flow, according to some embodiments of the present disclosure. Referring to fig. 1, the program testing method may include the steps of:
step S110, determining a test case of the application program, and executing the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image.
Step S120, determining the similarity between the reference image and the test image as a first similarity, and determining whether the first similarity is smaller than a first threshold.
Step S130, if the first similarity is smaller than a first threshold, image segmentation processing is respectively carried out on the reference image and the test image to obtain a block reference image and a block test image.
Step S140, determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block.
According to the program test method in the exemplary embodiment, on one hand, the image similarity calculation and image segmentation technology is applied to the program test, and the images with the similarity smaller than the threshold are segmented, so that the target abnormal blocks in the use case images can be accurately positioned. On the other hand, the similarity is compared with the threshold, and the position where the difference is generated between the test image and the reference image is positioned according to the comparison result, so that the manual intervention in the program test can be reduced, the labor is saved, and the test efficiency is improved. In another aspect, because manual intervention is reduced in the program testing process, certain manual errors can be avoided, and the testing quality of the program is ensured.
Next, the program test method in the present exemplary embodiment will be further explained.
In step S110, a test case of the application program is determined, and the test case is executed to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image.
In some exemplary embodiments of the present disclosure, the application program may be software written for some application purpose for the user, for example, the application software may include shopping software, news software, instant messaging software, and the like. A Test Case (Test Case) may be a set of Test inputs, execution conditions, and expected results tailored for a particular target to verify whether a particular software requirement is met. The contents include test targets, test environments, input data, test steps, expected results, test scripts, etc. The case image may be a result screenshot obtained after executing the test case. The reference image may be the first test image under the same test set or the test image of the last pass test. The test image may be an operation screenshot corresponding to the last execution of the test script. In the disclosed exemplary embodiment, the reference image may be regarded as a reference image, and the test image is compared with the reference image to determine whether the test image passes the test. The program test mode in the present disclosure is mainly used for front-end testing for application programs.
Referring to fig. 2, fig. 2 schematically shows an overall technical flow diagram of a program test method. In step S210, before performing a program test on an application program, a test case and a front-end automation test script of the application program may be determined according to the project requirement document. In step S220, a front-end automated test script is executed, and a case image corresponding to each test case is obtained by using the test case, where the case image may include a reference image and a test image, so as to compare similarity between the obtained reference image and the test image and determine whether the test case passes the test.
According to some exemplary embodiments of the present disclosure, an image storage directory corresponding to each test case is determined; determining a test script corresponding to the test case, and executing the test script to obtain a case image; and respectively storing the case images to the image storage catalogues corresponding to the test cases. The image storage directory may be a directory corresponding to a target position of a case image for storing each group of test cases. The test script may be a script program corresponding to the running test case.
Because a certain test version may have multiple groups of test cases in a test project, the image storage directories corresponding to the groups of test cases can be determined, so that case images obtained after running the test cases are stored in the corresponding image storage directories. After the test script corresponding to the test case is determined, the test script can be operated to operate the test case and obtain a case image, and the obtained case image is stored. For example, the image storage directory of the first group of Test cases of the Application program is "C: \ Users \ Application \2020\ Test 1", and the case images of the first group of Test cases can be directly stored in the image storage directory. In addition, subdirectories can be established under the storage directory, for example, a base subdirectory is established for storing the reference image, and a test subdirectory is established for storing the test image.
In step S120, the similarity between the reference image and the test image is determined as a first similarity, and it is determined whether the first similarity is smaller than a first threshold.
In some exemplary embodiments of the present disclosure, the first similarity may be a similarity between two images obtained by performing a similarity calculation on the reference image and the test image. The first threshold may be a pre-configured value for comparison with the first similarity.
Referring to fig. 2, in step S230, a similarity between the reference image and the test image may be calculated, the calculated first similarity is compared with a first threshold, and whether the test case passes or not is determined according to a comparison result.
According to some exemplary embodiments of the present disclosure, an image feature of a reference image is extracted as a reference image feature, and an image feature of a test image is extracted as a test image feature; the reference image features are compared to the test image features to determine a first similarity. The image features mainly comprise color features, texture features, shape features and spatial relationship features of the image. The reference image feature may be an image feature contained in the image-oriented image. The test image feature may be an image feature contained in the test image.
In the disclosure, an Image Retrieval technique may be used to calculate the similarity between the reference Image and the test Image, for example, a Content-based Image Retrieval (CBIR) technique may be used. The Similarity calculation method for calculating the Similarity between the reference image and the test image may include a Structural Similarity (SSIM) method, which is a fully-referenced image quality evaluation index and measures image Similarity in terms of brightness, contrast, and structure. The similarity calculation method may further include similarity calculation of histogram matching, for example, first calculating histograms, HistA and HistB, of the reference image and the test image, respectively, and then calculating normalized correlation coefficients of the two histograms, such as babbitt distance, histogram intersection distance, and the like. The similarity calculation method may also include image similarity calculation based on feature points, such as Scale-invariant feature transform (SIFT) method, and the like. The image retrieval technology can be adopted to respectively extract the reference image characteristics of the reference image and the test image characteristics of the test image, and then the first similarity is determined by comparing the similarity degree of the reference image characteristics and the test image characteristics.
In step S130, if the first similarity is smaller than the first threshold, image segmentation processing is performed on the reference image and the test image, respectively, to obtain a block reference image and a block test image.
In some exemplary embodiments of the present disclosure, the image segmentation process may be a technique and process that divides the image into several specific regions with unique properties and proposes an object of interest. The existing image segmentation methods mainly include the following categories: a threshold-based segmentation method, a region-based segmentation method, an edge-based segmentation method, a particular theory-based segmentation method, and the like. The block reference image may be a block image obtained by performing image segmentation processing on the reference image. The block test image may be a block image obtained by performing image segmentation processing on the test image.
After the first similarity is determined, the first similarity may be compared with a first threshold, and if the first similarity is greater than or equal to the first threshold, the similarity between the reference image and the test image of the test case may be considered to meet the test requirement, and the test result of the test case is determined to be a test pass. At this time, step S250 may be directly performed to review the comparison result and the test result and analyze whether the test result is accurate. If the first similarity is smaller than the first threshold, the test result of the test case is considered to be failed in the test, the difference between the test image and the reference image is large, and the difference position between the test image and the reference image can be further analyzed. The first similarity and the first threshold are compared and judged, so that manual judgment is replaced by threshold judgment, and manual intervention in the test process can be reduced.
Those skilled in the art will readily understand that, in different test scenarios, the specific value of the first threshold may be configured according to the specific test requirements of different test cases, and the present disclosure does not make any special limitation on the specific value of the first threshold. For example, for a page of a code restructuring class, since only the back-end code is restructured and optimized, the front-end page should not be changed, and thus, the first threshold may be set to 100%; for a page of the page reprint category, after the page reprint is performed, the positions of some function controls in the original page in the page may change, for example, the function control a in the upper right corner of a certain page moves to the lower right corner of the page after the page reprint. For this case, the page after page reprinting may be allowed to have a difference from the original page, for example, the first threshold may be set to 75%.
According to some exemplary embodiments of the present disclosure, a target number is determined; the target quantity is the quantity of images obtained after image segmentation processing; performing image segmentation processing on the reference image to obtain a target number of block reference images; and carrying out image segmentation processing on the test images to obtain the target number of block test images. The target number may be the number of images obtained by performing image segmentation processing on the reference image or the test image, for example, the target number may be 4, 6, 8, 16, 32, and the like, and in an actual test scenario, the target number may be set according to a test requirement. After the comparison result with the first similarity smaller than the first threshold is obtained, image segmentation processing can be respectively carried out on the reference image and the test image according to the target number so as to respectively obtain the block reference image and the block test image with the target number, and for the test image which does not meet the first threshold, the position where the difference is generated can be further reduced, so that a tester can be helped to judge the problem of the test.
Referring to fig. 3, fig. 3 schematically shows a use case diagram before image segmentation is performed on a use case image. The target number may be determined to be 4 according to the similarity calculation result, and in the case of the test image, the image segmentation process may be performed on the test image according to the segmentation lines 310 and 320 shown in the figure. Referring to fig. 4, fig. 4 schematically shows a result diagram after image segmentation processing is performed on a use case image. After the image segmentation processing is performed on the test image, the segmentation result of the test image into 4 block images, such as a block test image 1 (image 410), a block test image 2 (image 420), a block test image 3 (image 430), and a block test image 4 (image 440), can be obtained.
In step S140, a target difference block is determined according to the block reference image and the block test image, so as to verify a target test case corresponding to the target difference block.
In some example embodiments of the present disclosure, the target test case may be a test case corresponding to the target difference block. After the block reference image and the block test image are obtained, the block image with the difference between the block reference image and the block test image can be compared and used as a target difference block, so that a target test case corresponding to the target difference block can be verified, and the reason why the test fails is analyzed and determined.
According to some exemplary embodiments of the present disclosure, the similarity between each block reference image and each block test image is calculated respectively to obtain a plurality of second similarities; and comparing each second similarity with a second threshold value to determine target difference blocks according to the comparison result. The second similarity may be an image similarity value obtained by performing similarity calculation between each two of the block reference images and each block test image. The second threshold may be a pre-configured value for comparison with the second similarity. The target difference block may be a target block image determined from the block test images, in which the second similarity is smaller than a second threshold after the pairwise similarity comparison between each block reference image and each block test image. There may be one or more target difference blocks.
Referring to fig. 2, after the target number of block reference images and the target number of block test images are obtained, image similarities may be calculated between each two of the block reference images and the block test images, respectively, to obtain a plurality of second similarities in step S240. Assuming that the number of targets is N, in the present exemplary embodiment, the number of the obtained second similarities is N2And (4) respectively. The obtained N2And comparing the second similarity with a second threshold respectively, and judging the numerical value between the second similarity and the second threshold to obtain a comparison result, wherein the comparison result comprises the second similarity with the second similarity smaller than the second threshold. And determining target difference blocks from the block test images according to the comparison result.
According to some exemplary embodiments of the present disclosure, a first image number corresponding to each block reference image is determined, and a second image number corresponding to each block test image is determined; generating a plurality of similarity numbers according to the first image number and the second image number; determining second similarity corresponding to each similarity number, and judging whether the second similarity is smaller than a second threshold value; and determining a target similarity number with the second similarity smaller than a second threshold value, and determining the block test image corresponding to the target similarity number as a target difference block. The first image number may be an image number corresponding to each of a plurality of block reference images obtained by performing image division processing on a reference image. The second image number may be an image number corresponding to each of a plurality of block test images obtained by performing image segmentation processing on the test image. The similarity number may be a number corresponding to each of the plurality of second similarities, and the similarity number may be used to distinguish different second similarities. The target similarity number may be a number corresponding to a second similarity whose value is smaller than a second threshold value.
According to the first image numbers respectively corresponding to the block reference images and the second image numbers respectively corresponding to the block test images, the similarity numbers between the plurality of block reference images and the block test images can be generated according to the first image numbers and the second image numbers; and respectively calculating the magnitude relation between each second similarity and a second threshold, screening out the similarity number with the second similarity smaller than the second threshold, determining the similarity number as a target similarity number, and determining the block test image corresponding to the target similarity number as a target difference block. Similarity calculation is carried out between every two block reference images and every two block test images, failure of executing test cases caused by position revising of the front end of certain pages can be avoided, and the probability of misjudgment can be reduced to a certain extent.
For example, assuming that the target number is 4, the reference image and the test image may be respectively subjected to image segmentation processing to obtain 4 reference block images and 4 test block images, the images obtained after the image segmentation processing are respectively numbered, for example, the reference block images may be respectively numbered as S1, S2, S3 and S4, the test block images may be respectively numbered as T1, T2, T3 and T4, similarity values may be respectively calculated between the obtained plurality of block images, 16 second similarities may be obtained, and the second similarity may be expressed as Score (T2, T3 and T4)N，SN). The results of the calculated 16 second similarities are shown in table 1.
TABLE 1
| Degree of similarity | Image S1 | Image S2 | Image S3 | Image S4 |
| Image T1 | Score(T1,S1) | Score(T1,S2) | Score(T1,S3) | Score(T1,S4) |
| Image T2 | Score(T2,S1) | Score(T2,S2) | Score(T2,S3) | Score(T2,S4) |
| Image T3 | Score(T3,S1) | Score(T3,S2) | Score(T3,S3) | Score(T3,S4) |
| Image T4 | Score(T4,S1) | Score(T4,S2) | Score(T4,S3) | Score(T4,S4) |
According to the obtained similarity result table, the difference point between the reference image and the test image can be more accurately determined to be generated by which block image. The number of the target similarity determined in table 1 that the similarity value is smaller than the second threshold may be determined, and the block test image corresponding to the target similarity number may be determined as the target difference block.
According to some exemplary embodiments of the present disclosure, a target case image corresponding to a target test case is obtained; and determining an abnormal code segment of the target test case according to the target case image so as to adjust the abnormal code segment. The target use case image can be a use case image corresponding to the target test case, wherein the use case image comprises a target reference image and a target test image. The abnormal code segment may be a code segment corresponding to the failure of the test in the application program project code.
Referring to fig. 2, in step S250, a recheck is performed according to the image similarity comparison result and the test result obtained in steps S130 and S140 to determine a target case image corresponding to the target test case, an abnormal code segment causing the difference can be located by analyzing the difference between the target reference image and the target test image, the determined abnormal code segment is fed back to a tester, the tester can determine whether a problem (bug) exists by analyzing the abnormal code segment, and if the bug exists in the abnormal code segment, the abnormal code segment is adjusted in a targeted manner.
It should be noted that the terms "first", "second", and the like, used in the present disclosure, are only used for distinguishing different similarities, different thresholds, and different image numbers, and should not impose any limitation on the present disclosure.
In conclusion, the test case of the application program is determined, and the test case is executed to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image; determining the similarity between the reference image and the test image as a first similarity, and judging whether the first similarity is smaller than a first threshold value; if the first similarity is smaller than a first threshold value, respectively carrying out image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image; and determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block. According to the program testing method, on one hand, the image similarity calculation and image segmentation technology is applied to program testing, and the target abnormal blocks in the case images can be accurately positioned so as to verify the test cases corresponding to the target abnormal blocks. On the other hand, the similarity is compared with the threshold, and the difference position of the case image is determined according to the comparison result, so that the manual intervention in the program test can be reduced, the labor is saved, and the test efficiency is improved. In another aspect, because manual intervention is reduced in the program testing process, certain manual errors can be avoided, and the testing quality of the program is ensured. On the other hand, the program testing method can solve the problems of large workload of test case preparation, poor adaptation to demand change, high regression testing cost and the like in the front-end automatic test.
It is noted that although the steps of the methods of the present invention are depicted in the drawings in a particular order, this does not require or imply that the steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
In addition, in the present exemplary embodiment, a program test apparatus is also provided. Referring to fig. 5, the program testing apparatus 500 may include: a use case image determination module 510, a similarity contrast module 520, an image blocking module 530, and a difference blocking verification module 540.
Specifically, the use case image determining module 510 is configured to determine a test use case of the application program, and execute the test use case to determine a use case image corresponding to the test use case; the use case image comprises a reference image and a test image; the similarity comparison module 520 is configured to determine that the similarity between the reference image and the test image is a first similarity, and determine whether the first similarity is smaller than a first threshold; the image segmentation module 530 is configured to perform image segmentation processing on the reference image and the test image respectively to obtain a segmented reference image and a segmented test image if the first similarity is smaller than a first threshold; the difference blocking verification module 540 is configured to determine a target difference block according to the block reference image and the block test image, so as to verify a target test case corresponding to the target difference block.
The program testing device 500 may calculate a first similarity between a test image and a reference image corresponding to the test case, when the first similarity is smaller than a first threshold, perform image segmentation processing on the reference image and the test image respectively to obtain a block reference image and a block test image, calculate a second similarity between each two of the block reference image and the block test image, compare the obtained second similarity with a second threshold, determine a target difference block according to a comparison result, and verify a target test case corresponding to the target difference block to analyze a reason why the test does not pass, which may reduce manual intervention in a front-end test process, save labor cost, improve an automated test degree, and improve test efficiency; meanwhile, certain human errors can be avoided, the quality of the software project is guaranteed, and the device is an effective program testing device.
In an exemplary embodiment of the present disclosure, the use case image determining module includes a use case image determining unit, configured to determine an image storage directory corresponding to each test case; determining a test script corresponding to the test case, and executing the test script to obtain a case image; and respectively storing the case images to the image storage catalogues corresponding to the test cases.
In an exemplary embodiment of the present disclosure, the similarity comparison module includes a similarity determination unit for extracting an image feature of a reference image as a reference image feature and extracting an image feature of a test image as a test image feature; the reference image features are compared to the test image features to determine a first similarity.
In an exemplary embodiment of the present disclosure, the image blocking module includes an image blocking unit for determining a target number; the target quantity is the quantity of images obtained after image segmentation processing; performing image segmentation processing on the reference image to obtain a target number of block reference images; and carrying out image segmentation processing on the test images to obtain the target number of block test images.
In an exemplary embodiment of the present disclosure, the difference blocking verification module includes a difference blocking determination unit for calculating similarities between the respective blocking reference images and the respective blocking test images, respectively, to obtain a plurality of second similarities; and comparing each second similarity with a second threshold value to determine target difference blocks according to the comparison result.
In an exemplary embodiment of the present disclosure, the difference block determination unit includes a difference block determination subunit configured to determine a first image number corresponding to each of the block reference images, and determine a second image number corresponding to each of the block test images; generating a plurality of similarity numbers according to the first image number and the second image number; determining second similarity corresponding to each similarity number, and judging whether the second similarity is smaller than a second threshold value; and determining a target similarity number with the second similarity smaller than a second threshold value, and determining the block test image corresponding to the target similarity number as a target difference block.
In an exemplary embodiment of the present disclosure, the program testing apparatus further includes a code segment determining module, configured to obtain a target case image corresponding to the target test case; and determining an abnormal code segment of the target test case according to the target case image so as to adjust the abnormal code segment.
The details of each virtual program testing device module are described in detail in the corresponding program testing method, and therefore are not described herein again.
It should be noted that although in the above detailed description several modules or units of the program test device are mentioned, this division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
In addition, in an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above method is also provided.
As will be appreciated by one skilled in the art, aspects of the present invention may be embodied as a system, method or program product. Thus, various aspects of the invention may be embodied in the form of: an entirely hardware embodiment, an entirely software embodiment (including firmware, microcode, etc.) or an embodiment combining hardware and software aspects that may all generally be referred to herein as a "circuit," module "or" system.
An electronic device 600 according to such an embodiment of the invention is described below with reference to fig. 6. The electronic device 600 shown in fig. 6 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present invention.
As shown in fig. 6, the electronic device 600 is embodied in the form of a general purpose computing device. The components of the electronic device 600 may include, but are not limited to: the at least one processing unit 610, the at least one memory unit 620, a bus 630 connecting different system components (including the memory unit 620 and the processing unit 610), and a display unit 640.
Wherein the storage unit stores program code that is executable by the processing unit 610 such that the processing unit 610 performs the steps according to various exemplary embodiments of the present invention as described in the above section "exemplary method" of the present specification.
The storage unit 620 may include readable media in the form of volatile memory units, such as a random access memory unit (RAM)621 and/or a cache memory unit 622, and may further include a read only memory unit (ROM) 623.
The storage unit 620 may include a program/utility 624 having a set (at least one) of program modules 625, such program modules 625 including, but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which, or some combination thereof, may comprise an implementation of a network environment.
The electronic device 600 may also communicate with one or more external devices 670 (e.g., keyboard, pointing device, bluetooth device, etc.), with one or more devices that enable a user to interact with the electronic device 600, and/or with any devices (e.g., router, modem, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication may occur via an input/output (I/O) interface 650. Also, the electronic device 600 may communicate with one or more networks (e.g., a Local Area Network (LAN), a Wide Area Network (WAN), and/or a public network such as the Internet) via the network adapter 660. As shown, the network adapter 660 communicates with the other modules of the electronic device 600 over the bus 630. It should be appreciated that although not shown in the figures, other hardware and/or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, among others.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
In an exemplary embodiment of the present disclosure, there is also provided a computer-readable storage medium having stored thereon a program product capable of implementing the above-described method of the present specification. In some possible embodiments, aspects of the invention may also be implemented in the form of a program product comprising program code means for causing a terminal device to carry out the steps according to various exemplary embodiments of the invention described in the above-mentioned "exemplary methods" section of the present description, when said program product is run on the terminal device.
Referring to fig. 7, a program product 700 for implementing the above method according to an embodiment of the present invention is described, which may employ a portable compact disc read only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited in this regard and, in the present document, a readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
The program product may employ any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
A computer readable signal medium may include a propagated data signal with readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take many forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A readable signal medium may also be any readable medium that is not a readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device.
Program code embodied on a readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.
Program code for carrying out operations for aspects of the present invention may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., through the internet using an internet service provider).
Furthermore, the above-described figures are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the invention, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (10)
1. A program testing method, comprising:
determining a test case of an application program, and executing the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image;
determining the similarity between the reference image and the test image as a first similarity, and judging whether the first similarity is smaller than a first threshold value;
if the first similarity is smaller than the first threshold, respectively carrying out image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image;
and determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block.
2. The program testing method according to claim 1, wherein the application program includes a plurality of groups of test cases, and the executing the test cases to determine case images corresponding to the test cases includes:
determining an image storage directory corresponding to each test case;
determining a test script corresponding to the test case, and executing the test script to obtain a case image;
and respectively storing the case images to an image storage catalog corresponding to each test case.
3. The program testing method according to claim 1, wherein the determining that the similarity between the reference image and the test image is a first similarity comprises:
extracting the image features of the reference image as reference image features, and extracting the image features of the test image as test image features;
comparing the reference image feature with the test image feature to determine the first similarity.
4. The program testing method according to claim 1, wherein the performing image segmentation processing on the reference image and the test image to obtain a block reference image and a block test image comprises:
determining a target number; wherein the target number is the number of images obtained after the image segmentation processing;
performing image segmentation processing on the reference images to obtain the target number of block reference images;
and carrying out image segmentation processing on the test images to obtain the target number of block test images.
5. The program test method according to claim 1 or 4, wherein the determining a target difference block from the block reference image and the block test image comprises:
respectively calculating the similarity between each block reference image and each block test image to obtain a plurality of second similarities;
and comparing each second similarity with a second threshold value to determine the target difference blocks according to the comparison result.
6. The program testing method according to claim 5, wherein the comparing each of the second similarities with a second threshold to determine the target difference block according to the comparison result comprises:
determining a first image number corresponding to each block reference image, and determining a second image number corresponding to each block test image;
generating a plurality of similarity numbers according to the first image number and the second image number;
determining second similarity corresponding to each similarity number, and judging whether the second similarity is smaller than the second threshold value;
and determining a target similarity number with the second similarity smaller than the second threshold, and determining the block test image corresponding to the target similarity number as the target difference block.
7. The program testing method of claim 1, further comprising:
acquiring a target case image corresponding to the target test case;
and determining an abnormal code segment of the target test case according to the target case image so as to adjust the abnormal code segment.
8. A program test apparatus, comprising:
the case image determining module is used for determining a test case of the application program and executing the test case to determine a case image corresponding to the test case; the use case image comprises a reference image and a test image;
the similarity comparison module is used for determining that the similarity between the reference image and the test image is a first similarity and judging whether the first similarity is smaller than a first threshold value or not;
the image blocking module is used for respectively carrying out image segmentation processing on the reference image and the test image to obtain a blocked reference image and a blocked test image if the first similarity is smaller than the first threshold;
and the difference block verification module is used for determining a target difference block according to the block reference image and the block test image so as to verify a target test case corresponding to the target difference block.
9. An electronic device, comprising:
a processor; and
a memory having stored thereon computer readable instructions which, when executed by the processor, implement a program testing method according to any one of claims 1 to 7.
10. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out a program testing method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456655.5A CN113722203A (en) | 2020-05-26 | 2020-05-26 | Program testing method and device, electronic device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010456655.5A CN113722203A (en) | 2020-05-26 | 2020-05-26 | Program testing method and device, electronic device and computer readable storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113722203A true CN113722203A (en) | 2021-11-30 |
Family
ID=78672012
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010456655.5A Pending CN113722203A (en) | 2020-05-26 | 2020-05-26 | Program testing method and device, electronic device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113722203A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114116514A (en) * | 2021-12-03 | 2022-03-01 | 华人运通(上海)云计算科技有限公司 | User interface test acceptance method |
| CN115048309A (en) * | 2022-06-27 | 2022-09-13 | 广州掌动智能科技有限公司 | Non-intrusive APP software performance test method and system |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678105A (en) * | 2012-09-17 | 2014-03-26 | 百度在线网络技术（北京）有限公司 | Page code testing method and device |
| CN107566826A (en) * | 2017-01-12 | 2018-01-09 | 北京大学 | The method of testing and device of grating image processor |
| CN108920380A (en) * | 2018-07-20 | 2018-11-30 | 百度在线网络技术（北京）有限公司 | Test method, device, server, equipment and the storage medium of the software compatibility |
| CN109471805A (en) * | 2018-11-14 | 2019-03-15 | 网易（杭州）网络有限公司 | Resource testing method and device, storage medium, electronic equipment |
| CN109784369A (en) * | 2018-12-14 | 2019-05-21 | 深圳壹账通智能科技有限公司 | H5 page test method and device, electronic equipment, storage medium |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
| CN109949295A (en) * | 2019-03-21 | 2019-06-28 | 中国工商银行股份有限公司 | Otherness detection method, device and the computer storage medium of browsing device net page |
-
2020
- 2020-05-26 CN CN202010456655.5A patent/CN113722203A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103678105A (en) * | 2012-09-17 | 2014-03-26 | 百度在线网络技术（北京）有限公司 | Page code testing method and device |
| CN107566826A (en) * | 2017-01-12 | 2018-01-09 | 北京大学 | The method of testing and device of grating image processor |
| CN108920380A (en) * | 2018-07-20 | 2018-11-30 | 百度在线网络技术（北京）有限公司 | Test method, device, server, equipment and the storage medium of the software compatibility |
| CN109471805A (en) * | 2018-11-14 | 2019-03-15 | 网易（杭州）网络有限公司 | Resource testing method and device, storage medium, electronic equipment |
| CN109784369A (en) * | 2018-12-14 | 2019-05-21 | 深圳壹账通智能科技有限公司 | H5 page test method and device, electronic equipment, storage medium |
| CN109815156A (en) * | 2019-02-28 | 2019-05-28 | 北京百度网讯科技有限公司 | Displaying test method, device, equipment and the storage medium of visual element in the page |
| CN109949295A (en) * | 2019-03-21 | 2019-06-28 | 中国工商银行股份有限公司 | Otherness detection method, device and the computer storage medium of browsing device net page |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114116514A (en) * | 2021-12-03 | 2022-03-01 | 华人运通(上海)云计算科技有限公司 | User interface test acceptance method |
| CN115048309A (en) * | 2022-06-27 | 2022-09-13 | 广州掌动智能科技有限公司 | Non-intrusive APP software performance test method and system |
| CN115048309B (en) * | 2022-06-27 | 2023-03-07 | 广州掌动智能科技有限公司 | Non-intrusive APP software performance test method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP2019520615A (en) | Character recognition method, device, server and storage medium of claim document for damages | |
| US20110131551A1 (en) | Graphical user interface input element identification | |
| US20090273597A1 (en) | User interface screen layout analysis using hierarchical geometric features | |
| US20170139813A1 (en) | Method and device for checking influence of deleting cache file, and mobile terminal | |
| CN107992484B (en) | Method, device and storage medium for evaluating performance of OCR system | |
| CN110955590A (en) | Interface detection method, image processing method, device, electronic equipment and storage medium | |
| CN109872230B (en) | Test method and device of financial data analysis system, medium and electronic equipment | |
| CN112988557B (en) | Search box positioning method, data acquisition method, device and medium | |
| CN113722203A (en) | Program testing method and device, electronic device and computer readable storage medium | |
| US20200241900A1 (en) | Automation tool | |
| CN112015634A (en) | Page structure information generation method and device and electronic equipment | |
| CN113609028A (en) | Page testing method, computing device and readable storage medium | |
| CN110990285B (en) | UI (user interface) automatic testing method and device | |
| CN112395182A (en) | Automatic testing method, device, equipment and computer readable storage medium | |
| CN108228611B (en) | Document information copying method and device | |
| CN112667502A (en) | Page testing method, device and medium | |
| CN115905016A (en) | BIOS Setup search function test method and device, electronic equipment and storage medium | |
| CN111008140B (en) | Cross-platform UI (user interface) automatic testing method and device | |
| CN112003647B (en) | Detection method, device and equipment for recycled optical modem and storage medium | |
| CN110827261B (en) | Image quality detection method and device, storage medium and electronic equipment | |
| CN112286785B (en) | Abnormality detection method and device for user interface | |
| CN113704125A (en) | Black box automatic testing method based on graphical user interface | |
| CN113468076A (en) | Application program exception testing method, device, equipment and storage medium | |
| CN110083807B (en) | Contract modification influence automatic prediction method, device, medium and electronic equipment | |
| CN113673214A (en) | Information list alignment method and device, storage medium and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: Jingdong Digital Technology Holding Co.,Ltd. Address after: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant after: Jingdong Digital Technology Holding Co.,Ltd. Address before: Room 221, 2 / F, block C, 18 Kechuang 11th Street, Daxing District, Beijing, 100176 Applicant before: JINGDONG DIGITAL TECHNOLOGY HOLDINGS Co.,Ltd. |