JP2005100068A - Automatic software verification system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an automatic software verification system for reducing man-hours required to create a regression test program necessary during development as a whole. <P>SOLUTION: In an automatic software verification method which supports a creation of a regression test method of software created during development of the software for use in products a part of the functions of which is achieved by the software, a state of each function of the products is defined (#100: S102-S112), the defined functions are associated with names of memory data RAMs of a micro processor to control the functions of the products (S108, S110), the associated functions are converted into absolute addresses of memory data and executes regression tests for the products (#150, #200), and the memory data of the absolute addresses indicating test results obtained after the regression tests are inversely converted into the functions (S206). <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a software automatic verification system that supports creation of a software regression test method created when developing a new product in a product in which a part of the functions is realized by software.

  In recent years, in many products configured using electronic devices, some or most of the functions are realized by software using a microcomputer. Therefore, when a new product is developed, the function of the new product as a whole is often realized by creating software including the function of the conventional product and the new function. In other words, the development of a new product can be said to be realized by upgrading the software that realizes the functions of the conventional product together with the development of hardware.

  Prior to commercialization, as part of evaluating whether the new product that will be developed will function as designed, the newly created program will be checked for errors and unexpected effects due to software changes. A regression test is performed.

  In the regression test, the entire function of the new product is of course the target. Specifically, the developer sets all the verification items of the software that realizes the functions of the new product, and for each verification item, the parameter stored in the specific address of the specific RAM existing in the microcomputer. Methods are known for performing regression tests by verifying values. In this method, the regression test function is also mainly realized by software, that is, a program, and the regression test is executed by executing the program. If a failure is found as a result of the regression test, a countermeasure is applied to the software, and a regression test is performed again on the countermeasured software. Then, countermeasures and regression tests are repeatedly performed until no defect is found.

  Conventionally, as a means for supporting a regression test, an automatic test apparatus has been proposed that enables an automatic test of a window creation means in which the display content of a window displayed on a monitor is changed in response to a keyboard or mouse input. (Patent Document 1). In the automatic test apparatus, the event generating means creates an event signal based on input signals from the keyboard and mouse. Then, the event storage means stores the event signal. It has been visually confirmed in advance by the user that the window created by the window creating means in response to the event signal is in the correct state. This is stored in the frame buffer, and image data is created by the image data creation means. On the other hand, event signals are reproduced by the event reproduction means in the same order as the image control means. A window is created by the window creation means using the reproduced event signal as an input, the result obtained by the image comparison means is compared with the correct window value, and a test result is output.

  Furthermore, a software test apparatus has been proposed as a means for quickly and efficiently performing a test of upgraded software and a detection of a degradation bug (Patent Document 2). In the test apparatus, software before upgrading (conventional software) is installed in a computer system, and an inspector performs a test using a keyboard or a mouse. The test result is displayed on the display unit via the converter. The obtained test procedures and test results are stored as electronic files in a computer system via a converter. If a degradation bug is detected, bug related information collected by the computer system is stored in the computer system. Thus, the initial software test contents are learned. When performing the same test with the upgraded software installed in the computer system, the test procedure that is the learning result is read and an automatic test is performed, and this test result should also be compared with the test result filed above. Check with. Bug related information is also used when a bug is detected.

  In addition, as a means for pursuing the cause of a degradation bug detected by testing the upgraded software, a software automatic test method has been proposed (Patent Document 3). In this automatic test method, the trace information of the system before the upgrade and the trace information of the system after the upgrade are compared, the part where the function call order or the argument value is different is found, and the module change list Predict whether the module is causing degradation. That is, each time a function is called when the system is tested, the name of the module to which the function belongs, the name of the function, the value of the input factor, the value of the output factor, and the return value are recorded as one trace. Furthermore, the creation date for each module when the test is executed is stored. Similarly, when the system is upgraded, the trace information and the creation date for each module are stored. Compare the trace information with the trace information from the previous test and check for differences.

  In order to investigate the cause of the difference, the user is shown whether or not both modules have been changed based on the creation date of the module that called the function and the creation date of the called module. In addition, the user is notified of whether the input argument, the output argument, and the return value at the time of the call are changed from the trace information. From this information, the user is informed whether the module that called the function is operating differently, the module that called the function is operating differently, or another module is operating differently. . This allows the user to predict where the degradation will occur.

  Further, a part to be read of the function belonging to the first module is searched from the trace information, the function belonging to the first module is called in the order of recording the trace information, and the value of the input argument is passed. The output argument value and return value, which are the function execution results, are compared with the values stored as trace information. This allows unit testing of the first module.

  Furthermore, even when testing with test data that is different from the test data when the trace was stored, the input argument value, output argument value, and return value are naturally different, but the function call order is often the same. . Therefore, the cause of the degradation can be pursued by comparing only the calling order of the functions. The function call order information is extracted from the stored trace information, the two information are compared, and the location where the function call is different is searched for and presented to the module that called the function. Allows the user to determine that there is a cause of

As described above, an automatic test apparatus for a window creation program whose display changes in response to a user's operation input (Patent Document 1), a test apparatus for a program for testing upgraded software and detecting a degradation bug (Patent Document 2) And a software automatic test method (Patent Document 3) used for estimating the cause of a degrading bug detected by testing the upgraded software.
JP-A-5-143269 JP-A-8-212108 JP-A-10-320234

  Even in the new product, most of the functions are the same as the existing functions, and the common functions have already been verified. In this sense, the test item to be verified when developing a new product is the function of the newly added software and the consistency between the software and the software that realizes the conventional function.

  However, it is very difficult to clearly separate software that realizes new functions from software that realizes conventional functions. Therefore, the developer is forced to set all the verification items of the software that realizes the functions of the new product individually, and to execute a regression test on the set verification items. Therefore, there is a waste that verification is repeatedly performed even on items that have already been verified.

  Furthermore, in recent years, even for similar products, the contents of function development using a microcomputer for each different model are rapidly becoming more sophisticated and complicated. Along with that, the development man-hours are also increasing rapidly, and the development capacity cannot keep up with the development load. In addition, to ensure product quality, thorough software verification at the time of development is necessary, but the increase in development man-hours is an obstacle.

  In particular, in the regression test, the parameter value stored at a predetermined address of a specific RAM in the product control device is used for every verification item of the software that realizes the function of the new product set by the developer. By verifying, it is determined whether there is any degradation in the upgraded software. In other words, in the regression test of new products, regarding newly added functions, the RAM address and parameters and parameter values corresponding to the new functions are specified and the specified values are verified by the upgraded program. You must create a regression test program. Furthermore, the RAM address and parameter corresponding to the conventional function and the value thereof are also changed by setting the function corresponding to the function of the new product at the time of version upgrade.

  In order to detect the presence / absence of the degradation, it is necessary to accurately grasp the RAM address that changes due to software upgrade and the normal value of the parameter value corresponding to the address, and determine the parameter value of the RAM address. Need to create a program. Such program creation requires very advanced knowledge and usually requires a specialized programmer. However, specialized programmers do not have knowledge about the contents to be verified of the functions of old and new products. On the other hand, the product developer knows the relationship between functions and parameters and the test items to be verified, but is skilled enough to create a program that reads the value of a specific RAM address based on the relationship. Few people have

  Therefore, the developer of the product needs to explain to the programmer about the function of the product determined by the programmer, and the programmer needs to create a program according to the received explanation. This process is also inefficient due to communication problems between developers and programmers and the skills of each other, which can take a lot of effort and time to complete a correct regression test program. It is. Therefore, there is a need for a support system that allows a developer who is familiar with the content of required functions for a new product to create a regression test program alone without the assistance of a specialized programmer.

  In the automatic test apparatus (Patent Document 1) for the window creation program described above, the keyboard and mouse input signals and the data displayed in the window are stored, and the keyboard and mouse input signals are reproduced and created. The data displayed in the window is compared with the data displayed in the stored window, and the test result is output. That is, it is only a window display program whose display contents are changed in response to an event. It does not support the creation of regression test programs that detect degradation.

  In the above-described program test apparatus (Patent Document 2), the presence or absence of a degradation bug is detected by a tool. However, even software that has been upgraded cannot be used for software that has been added with a new function because the same test procedure and test results as before the upgrade are automatically used. Further, it does not have means for appropriately setting parameters and RAM addresses for additional functions due to version upgrade, and does not support the creation of a regression test program for detecting degradation.

  Furthermore, in the software automatic test method (Patent Document 3), the user can compare the previous test trace information with the current test trace information and present the user with information on abnormal operation of the module. Supports prediction of degradation points. However, it does not have means for appropriately setting parameters and RAM addresses for additional functions due to version upgrades, and does not support the creation of a regression test program for detecting degradation.

  Therefore, the present invention automatically sets items for functions that have already been verified, and allows the developer to individually set only items for new functions. An object of the present invention is to provide an automatic software verification system that can reduce the man-hour required for creating a test program.

  A software automatic verification method for supporting creation of a software regression test method created when developing software used for a product in which a part of the function is realized by software, and defining the state of each function of the product A step of associating the defined function with a memory data RAM name of a microcomputer that controls the function of the product, a step of converting the associated function into an absolute address of the memory data, and a step of executing a regression test on the product And inversely converting memory data at an absolute address representing a test result obtained after the regression test into a function.

  In the present invention, even if the corresponding RAM address and value change according to the change of the software, which is performed for the addition or change of the software accompanying the change of the specification, or the software implemented for performance improvement, when the model of the product is changed. The same state name is displayed to the user. Therefore, the user can create a regression test method, execute the test, evaluate the test result, and further edit the regression test based on the evaluation result, even if the user does not have a high degree of program skill. Therefore, when a regression test method is set for a certain product, most of the regression test methods of the conventional model can be used for the subsequent regression test methods for new models. In other words, the regression test of the new model can be supported by adding the specification addition part of the new model to the regression test method of the conventional model. As a result, the cost can be reduced including the time, labor, and cost required for the regression test.

(First embodiment)
Hereinafter, embodiments of the present invention will be described with reference to FIGS. 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, and 12. Such a software automatic verification system will be described. In the present embodiment, it is possible to manually create a state transition table described later.

  As shown in FIG. 1, the software automatic verification system Svs1 according to the present embodiment is roughly divided into an inspection item setting device 100A and a verification target device 200. The verification target device 200 has a function that controls the function of a verification target device (not shown) that is a target of operation verification in the software automatic verification system Svs1, and is preferably configured by a microcomputer.

  The inspection item setting device 100A generates a verification command signal S31 having a verification command for verifying the operation of the verification target device 200, and outputs the verification command signal S31 to the verification target device 200.

  Based on the verification command signal S31, the verification target device 200 performs a regression test on its own function and detects the state of each function. The verification target device 200 further generates a verification target device state information signal S43 representing the verification target function state information corresponding to the detected functional state, and outputs it to the inspection item setting unit 100.

  The verification target device 200 includes a command receiver 201, a controller 203, a device control memory 205, and a data receiver 211. The device control memory 205 is a rewritable memory used by the verification target device 200 for controlling the verification target device, and includes a key correspondence memory writer 207 and a state correspondence memory reader 209.

  The command receiver 201 extracts the verification command S33 from the verification command signal S31 input from the inspection item setting unit 100A and outputs it to the controller 203. The verification command S33 is a signal including essentially the same information although the voltage level is different from that of the verification command signal S111 output from the controller 110A described later.

  The controller 203 generates a memory write instruction signal S35 and a memory read instruction signal S31 based on the verification command S33, and outputs them to the device control memory 205. In the device control memory 205, the key-corresponding memory writer 207 operates the operation key (for example, CD PLAY key) of the inspection target device corresponding to the key at a predetermined address of the device control memory 205 based on the memory write instruction signal S35. Etc.) is written. Further, the state correspondence memory reader 209 reads information indicating the state of the verification target device from a predetermined address in the device control memory 205 based on the verification command signal S31, and the controller 203 reads the information as a state memory data signal S38. Output to.

  Based on the state memory data signal S38, the controller 203 extracts the address value of the memory corresponding to the function and the data stored in the same address, and outputs it to the data receiver 211 as the state memory data S41. The data receiver 211 generates a state memory data transmission signal S43 having a format suitable for the transmission path between the verification target device 200 and the inspection item setting device 100A.

  As shown in FIG. 2, the inspection item setting device 100A includes a display device 101, an operation device 103, an operation order setting device 105, an operation order storage device 106, an expected value storage device 107, a comparator 109, a command transmitter 111, data It includes a receiver 113, a state correspondence memory storage 117, a target state name converter 119, a state name conversion table manual creation register 120, and a controller 110A. The controller 110A is connected to all the components of the inspection item setting device 100A and exchanges various signals with each other to control the operation of these components and the inspection item setting device 100A.

  Based on the image signal S101 supplied from the controller 110A, the display device 101 presents various information to the user and options that the user can instruct the inspection item setting device 100A in video or sound. The display device 101 is preferably composed of a display monitor and a speaker.

  The operation device 103 receives an operation by the user and generates an operation signal S103 indicating the content of the operation. The operation signal S103 is input to the controller 110A. Note that the controller 110A generates various signals including information necessary for processing in each component based on the operation signal S103, and outputs the signals.

  Based on the operation order request signal S135 supplied from the controller 110A, the operation order setting unit 105 generates an operation information signal S136 indicating the operation order of the device to be verified that the user desires for the inspection item setting unit 100A. . The operation information signal S136 is input to the controller 110A.

  The expected value storage unit 107 exchanges the controller 110A with the operation / data signal S107, generates expected value information I107, and outputs it to the comparator 109.

  The state name conversion table manual creation / registration device 120 generates state name conversion table information I120 based on the conversion table manual creation command signal S120 input from the controller 110A, and outputs the state name conversion table information I120 to the state correspondence memory storage device 117. To do.

  The state correspondence memory storage 117 is under verification based on the state name conversion table selection signal S117 input from the controller 110A and the state name conversion table information I120 input from the state name conversion table manual creation register 120. State name conversion table information I117 corresponding to the mode is generated and output to the target state name converter 119. The state name conversion table information I117 corresponding to the mode being verified may be input / output to / from the state correspondence memory storage 117 via an external recording medium such as a magnetic recording disk.

  The target state name converter 119 is based on the verification data signal S119 input from the controller 110A and the state name conversion table information I117 corresponding to the mode under verification input from the state correspondence memory storage 117. The state name information I119 is generated and output to the execution result storage unit 115.

  The execution result storage unit 115 generates execution result information I115 based on the execution result input / output signal S115 input from the controller 110A and the state name information I119 input from the converter 119 to the target state name. And output to the comparator 109.

  The comparator 109 compares the expected value information I107 input from the expected value storage 107 with the execution result information I115 input from the execution result storage 115, and generates a comparison result output signal S109 corresponding to the comparison result. Output to the controller 110A.

  The data receiver 113 generates a verification data signal S113 representing verification data based on the verification target device state information signal S43 input from the verification target device 200, and outputs the verification data signal S113 to the controller 110A.

  The command transmitter 111 generates a verification command signal S31 based on the verification command signal S111 input from the controller 110A and outputs the verification command signal S31 to the verification target device 200.

  The operation of the software automatic verification system Svs including the operation output of the inspection target device will be described with reference to FIGS.

  First, the main operation of the software automatic verification system Svs will be described with reference to the main flowchart shown in FIG. The operation of the automatic software verification system Svs is as follows: “State name definition” subroutine at step # 100, “Operation order setting” subroutine at step # 150, “Execution data generation” subroutine at step # 170, “State correspondence” This is realized by executing the “memory data generation” subroutine, the “execution data editing” subroutine in step # 250, and the “execution and result display” subroutine in step # 270.

  In the “state name definition” subroutine at step # 100, the state name of the function to be verified is defined.

  In the “operation order setting” subroutine in step # 150, the verification order is set.

  In the “execution data generation” subroutine of step # 170, data used for executing verification is generated.

  In the “state-corresponding memory data generation” subroutine of step # 200, data to be recorded in a state-corresponding memory storage (not shown) is generated corresponding to each state of the function to be verified.

  In the “execution data editing” subroutine of step # 250, the execution data generated in the “execution data generation” subroutine of step # 170 is edited as necessary.

  In the “execution and result display” subroutine of step # 270, the verification is executed and the verification result is displayed to the user.

  As shown in FIG. 4, in the state name definition subroutine of # 100, first, in step S102, the controller 110A generates an image signal S101 for rendering a list of processes that the user can select to define the state name. Generate. Then, based on the image signal S101, the display device 101 displays a “state name definition menu” to prompt the user to input a selection. The state name definition menu includes “state data registration”, “state name designation”, “state data RAM address designation”, “specific value designation of state data”, and “registration complete”. This menu is always displayed on the display unit 101 during execution of this subroutine, and the user can freely select and input it. After the elapse of a predetermined time sufficient for the user to operate the operation device 103 to select and input the “state name definition menu”, the control proceeds to the next step S104.

  In step S104, controller 110A determines whether or not the user has selected “status data registration” based on operation signal S103. If “status data registration” is not selected, it is determined No, control returns to step S102 described above, and the processes of steps S102 and S104 are repeated. If "Register status data" is selected, the determination is Yes and control proceeds to the next step S106.

  In step S106, designation of a state name to be registered as state data DS corresponding to a state to be verified, which is performed on the “state name designation” menu by the user using the operation device 103, is accepted. Then, the control proceeds to the next Step S108.

  In step S108, the name of the state data RAM in the device control memory 205 corresponding to the state name accepted in step S106, which is performed on the “state data RAM name designation” menu by the user using the operating device 103. Is accepted. Then, the control proceeds to the next Step S110.

  In step S110, specification of a value to be stored at an address corresponding to the name of the state data RAM specified in step S108, which is performed on the “specify specific value of state data” menu by the user using the operation device 103, is accepted. It is done. FIG. 8 shows an example of the status data DS. As shown in the figure, the state data DS stored in the state correspondence memory storage is expressed as a kind of table. The state data DS includes a plurality of state names NS indicating different states and a state value VS indicating each state. The state value VS is stored as a binary value in the reference memory.

  Specifically, in this example, the state name NS is “power OFF”, “power ON (STOP)”, “PLAY”, and “PAUSE”, and “00000000” and “10000000” are the state values VS for each. ”,“ 10001000 ”, and“ 100001100 ”constitute a state data DS.

  That is, in step S110, the state value VS is specifically designated for at least one of the above-described five kinds of state names NS. Then, the control proceeds to the next Step S112.

  In step S112, controller 110A determines based on operation signal S103 whether the user has selected a “registration complete” menu meaning that registration of status data DS is complete. If it is not selected, it is determined No, the process returns to step S102 described above, and after repeating the processes of steps S102 to S110, the process of this subroutine is performed when it is determined Yes in step S112. Exit. As described above, in this subroutine, after each state of the function for performing the regression test is defined, the control proceeds to the “operation order setting” subroutine of step # 150.

  In the “operation order setting” subroutine of step # 150, first, in step S2, the controller 110A generates an image signal S101 for drawing a list of processes that the user can select for setting the operation order. Then, based on the image signal S101, the display device 101 displays an “operation storage menu” to prompt the user to make a selection input. FIG. 9 shows a display example of the operation storage menu. The operation memory menu IS includes selection items represented by four types of key icons of “POWER”, “PLAY”, “PAUSE”, and “STOP”, “memory start”, “memory end”, “file save” "And" Execution screen "selection items are included. This menu is always displayed on the display unit 101 during execution of this subroutine, and the user can freely select and input it. After the elapse of a predetermined time sufficient for the user to operate the operation unit 103 and select and input the operation storage menu IS, the control proceeds to the next step S4.

  In step S4, controller 110A determines whether or not “start storage” is selected based on operation signal S103. Only when it is selected, the control proceeds to the next step S6.

  In step S6, the operation order setting unit 105 determines whether any of the four types of key icons in the operation storage menu IS is selected based on the operation signal S103 input via the controller 110A. When it is determined No, the process according to this step is repeated. Only when the determination is Yes, control proceeds to the next step S8.

  In step S8, the operation order setting unit 105 uses the operation information IO (not shown) indicating the type of the key icon determined to be selected in step S6 and the selected timing based on the operation signal S103 (S107). Some operation information S106 is generated and stored. Then, control proceeds to the next step S10.

  In step S <b> 10, the operation order setting unit 105 determines whether or not “end of storage” has been selected based on the operation information IO. In No, control returns to above-mentioned step S6. Then, after step S8, in step S10 again, it is determined whether or not “end of storage” has been selected. In this way, the operation information IO is added with the target selected by the user and the order selected at the selected timing. The stored operation information IO is later returned to the controller 110A as an operation information signal S136 in response to an operation order request signal S135 input from the controller 110A. Then, the process of this subroutine is finished. The control proceeds to the “execution data generation” subroutine in the next step # 170. That is, in this subroutine, after the order of executing the regression test is determined, “execution data generation” in step # 170 is started.

  In the “execution data generation” subroutine of step # 170, first, in step S12, the controller 110A determines whether or not “save file” is selected based on the operation signal S103. The determination in this step is repeated until it is determined Yes. When it is determined Yes in this step, the control proceeds to the next step S14.

  In step S14, the controller 110A outputs an operation order request signal S135, and reads the operation information IO generated in step S10 (# 150) from the operation order setting unit 105 as the operation information signal S136. Then, the operation information IO included in the operation information signal S136 is stored in the operation order storage unit 106 as a file having a predetermined file name. Then, the control proceeds to the next Step S16.

  In step S16, the controller 110A determines whether or not the “execution screen” is selected based on the operation signal S103. In the case of Yes, the control proceeds to the next step S18.

  In step S <b> 18, the controller 110 </ b> A generates an image signal S <b> 101 that renders a list of processes that the user can select for generating execution data. Based on the image signal S101, the display device 101 displays an “execution menu” to prompt the user to make a selection input. FIG. 10 shows a display example of the “execution menu”. The execution menu IE includes selection items of “execution file setting”, “expected value file designation”, and “execution”. This menu is always displayed on the display unit 101 during execution of this subroutine, and the user can freely select and input it. After the elapse of a predetermined time sufficient for the user to operate the operation device 103 and select and input the execution menu IE, the control proceeds to the next step S20.

  In step S20, controller 110A determines whether or not “execute file designation” is selected based on operation signal S103. If it is selected, the determination is Yes, and the control proceeds to the next step S22.

  In step S22, the execution file designated in step S20 is read from the operation order storage unit 106 to the controller 110A, and execution data DE is generated. Then, the control proceeds to the next Step S24.

  In step S24, controller 110A determines whether or not “execute” is selected based on operation signal S103. If “execute” is selected, the determination is Yes, and the control proceeds to the “state-corresponding memory data generation” subroutine of step # 200. As described above, after the execution file for executing the regression tests in the order determined in step # 150 is completed, the subroutine # 200 that executes the regression test and generates the state-corresponding memory data as a result thereof is provided. Be started.

  In the “state-corresponding memory data generation” subroutine of step # 200, first, in step S202, the controller 110A transmits a verification command signal S31 via the command transmitter 111. That is, the microcomputer RAM memory data (S111) corresponding to the key is output as the verification command signal S31 to the verification target device 200 at the timing designated by the target microcomputer. That is, it instructs the target microcomputer which bit of which address of the microcomputer RAM is to be changed.

  Specifically, first, the contents designated by the flag name of the microcomputer RAM memory are read from the operation order storage unit 106. Then, the value corresponding to the read content is mounted on the absolute address of the microcomputer RAM memory via the name conversion table, which is called an OUT file created by compiling the microcomputer software. Is performed. Thereby, for example, the “PLAY key ON flag” is converted into the absolute address “052A (hexadecimal) bit2” of the microcomputer RAM, and the process of setting the bit to “1” is executed. Then, the control proceeds to the next Step S204. Then, the control proceeds to the next Step S204.

  In step S204, the controller 110A sends a command (not shown) for reading the state RAM memory data in the microcomputer RAM to the state memory memory 117, and the memory data in the microcomputer RAM, for example, shown in FIG. The contents of the reference memory (X′0001) are received and stored in the state correspondence memory storage 117 of the inspection item setting device 100A. Then, the control proceeds to the next Step S206. As for the reference memory (X′0001), the conversion table to the absolute address is used as in the “PLAY key ON flag”.

  In step S206, the state correspondence memory data D117 stored in step S204 is converted into the target state name D119 by the target state name converter 119. For example, if the content of the reference memory (X′0001) is “00000000”, it is converted to “power OFF”. Then, the control proceeds to the next Step S208.

  In step S208, the controller 110A determines whether or not the operation storage data has been executed. When it is determined that the process has not been completed, the control returns to step S202 described above, and after the processes of steps S204 and S206, the process is terminated when the result of this step is Yes (execution is completed). In this subroutine, after the regression test is executed in the order and content determined in the above subroutine, the process proceeds to the “execution data editing” subroutine in step # 250.

  In the “execution data editing” subroutine of step # 250, first, in step S26, the controller 110A generates the state name D119 obtained in step S206 and the image signal S101 representing the key name, and the display 101 Output to. Based on the image signal S101, the display unit 101 displays an “edit menu” representing the key name and the obtained target state name D119, and prompts the user to make a selection input. FIG. 11 shows an example of the edit menu. The edit menu IM includes an acquisition state display table T that represents the selected key and the obtained state, and selection items of “edit”, “register as expected value”, and “execution screen”. Then, the control proceeds to the next Step S28.

  In step S28, based on the operation signal S103, the controller 110A determines whether or not “edit” has been selected. If selection of the “edit” key is detected, “Yes” is determined, and control proceeds to step S29.

  In step S <b> 29, the user's editing work on the content of “obtained state” displayed in the acquisition state display table T is accepted. Editing is performed by the user operating the operation device 103. Control returns to step S28 described above.

  On the other hand, if “edit” is not selected in step S28, it is determined No and control proceeds to step S30.

  In step S30, controller 110A determines whether or not “register as expected value” is selected based on operation signal S103. If it is not selected, it is determined as No, and the control returns to step S28 described above, and the process in step S29 or step S30 is repeated. Then, only when “Register as expected value” is selected, the determination is Yes, and the control proceeds to the next step S32.

  In step S32, the “obtained state” after editing is saved with a file name. Then, the control proceeds to the next Step S34.

  In step S34, based on the operation signal S103, the controller 110A determines whether or not the “execution screen” is selected. Only when selection of the “execution screen” key is detected, control proceeds to the next step S36.

  In step S <b> 36, the controller 110 </ b> A generates again the image signal S <b> 101 for rendering the execution menu IE illustrated in FIG. 10 and outputs the image signal S <b> 101 to the display device 101. The display device 101 displays an execution menu IE and prompts the user to perform a selection operation. After the elapse of a predetermined time sufficient for the user's selection input operation, the control proceeds to the next step S38.

  In step S38, the controller 110A determines whether or not the “execute file designation” key is selected based on the operation signal S103. Only when the selection of the “execute file designation” key is detected, the determination is Yes, and the control proceeds to the next step S40.

  In step S40, as in step S22 described above, the execution file is read and the execution data DE is generated. Then, the processing in this subroutine is completed, and the control proceeds to the “execution and result display” subroutine in the next step # 270.

  In the “execution and result display” subroutine of step # 270, first, in step S42, similarly to step S24 described above, controller 110A determines whether or not the execution key has been selected based on operation signal S103. . If selection of the execution key is not detected, control proceeds to step S44.

  In step S44, it is determined whether or not “expected value file designation” is selected. In No, control returns to above-mentioned step S42. On the other hand, if the determination is Yes, control proceeds to step S46.

  In step S46, the specified value file is read by the expected value storage unit 107 and set as an expected value. Then, the control proceeds to step S42 described above.

  On the other hand, when the selection of the execution key is detected in step S42, the determination is Yes, and the control proceeds to the “state corresponding memory data generation” subroutine of step # 200. In the # 200 subroutine, after the above-described processing, the control proceeds to the next step S48.

  In step S48, controller 110A determines whether or not an expected value file is specified. If the expected value file is not specified, it is determined No and control returns to step S26 described above. Then, the processes in steps S26 to # 200 described above are repeated. Then, when it is determined Yes in this step, the control proceeds to the next step S50.

  In step S50, the comparator 109 obtains the state obtained for each target key, the obtained state name (information ID 115 output from the execution result storage unit 115), and the expected value file state name (expected value storage unit). The information ID 107) output from 107 is compared. A comparison result signal S109 representing the comparison result is generated and output to 110A. Then, the control proceeds to the next Step S52.

  In step S52, the controller 110A generates an image signal S101 for rendering the comparison result based on the comparison result signal S109 and outputs the image signal S101 to the display device 101. FIG. 12 illustrates a comparison result screen IRC displayed on the display device 101. In the comparison result screen IRC, “obtained state”, “expected value”, and “comparison result” corresponding to each of the “selected keys” are developed in a matrix. In the example illustrated in FIG. 12, “PLAY”, “PAUSE”, “PLAY”, “STOP”, and “POWER” are selected, and the “obtained state”, “expected value”, and “ “Comparison result” is displayed. In the same example, the fact that the comparison result of the third “PLAY” from the top is “NG” is displayed together with a shadow.

  FIG. 8 shows an example of the state data DS displayed in step S110 described above.

  FIG. 9 shows an example of the operation storage menu IS displayed in the above-described step S2.

  FIG. 10 shows an example of the execution menu IE displayed in step S18 described above.

  FIG. 11 shows an example of the edit menu IM displayed in step S26 described above.

  FIG. 12 shows an example of the IRC displayed in step S52 described above.

  As described above, in this embodiment, the function state of the product is defined in association with the memory data for the microcomputer of the regression program using a name that is easy for the user to recognize, like the RAM name on the software. The defined name is converted into an absolute address where the memory data is stored. In this way, the function and the memory data in the RAM are associated with each other, and the product is made to function as an interface between the hardware and the user. As a result, the same state name is displayed to the user even if the corresponding RAM address and value change according to the change in the product model or the software.

  Further, by presenting the value of the RAM memory data after the regression test as the state of the product, the user can evaluate the regression test result without requiring a high degree of programming skill. Furthermore, based on this evaluation, a regression test item for a new function can be additionally set as a state name.

  In the present invention, a known technique can be used for conversion from a RAM name for a microcomputer to a RAM address. However, the conversion from the RAM value to the state after the regression test can be realized by a method uniquely proposed in the present invention. This is because it is necessary to refer to the contents of a plurality of microcomputer RAMs in order to see the state of the product after the test. For example, in this embodiment, for simplification of explanation, the PLAY state and the STOP state are viewed in one RAM area, but in practice, it is generally viewed by combining the contents of a plurality of memories. Is necessary.

(Second Embodiment)
The automatic software verification system according to the second embodiment of the present invention will be described below. In the present embodiment, a transition table can be automatically created as will be described later. In the automatic verification system Svs2 according to the present embodiment, the inspection item setting unit 100A is replaced with an inspection item setting unit 100B in the software automatic verification system Svs1 according to the first embodiment shown in FIG. Other than that, it is configured and operates in the same manner as the software automatic verification system Svs1. Therefore, only features unique to the software automatic verification system Svs2 will be described mainly.

  The inspection item setting device 100B will be described with reference to FIG. In the inspection item setting device 100B, in the inspection item setting device 100A, the controller 110A is replaced with the controller 110B, and an input device 132 from the spreadsheet application and an output device 134 to the spreadsheet application are added. .

  The input device 132 from the spreadsheet application outputs test item data S132 representing the verification test items and procedures executed in the verification target device 200 in the data format of the spreadsheet application to the controller 110B.

  The display device 101 presents the test items and the order to the user based on the test item data S132.

  The user can edit the test item data S132 by operating the operation unit 103 with respect to the test items and the order presented on the display unit 101.

  The output unit 134 for the spreadsheet application includes the edited test item data S132 or the expected value data and the execution result data including the data of the result of the inspection execution in the data format of the spreadsheet application in the controller 110B. Output to.

  Main operations of the controller 110B will be described with reference to FIGS. First, as shown in FIG. 14, the main flow of the controller 110B is the “operation procedure” of step # 300 before the “state name definition” subroutine of step # 100 of the main flow of the controller 110A shown in FIG. An “input” subroutine is provided. The “operation order setting” subroutine in step # 150 and the “execution data generation” subroutine in step # 170 are replaced with the “target state data generation” subroutine in step # 400.

  Furthermore, the “state-corresponding memory data generation” subroutine in step # 200 is deleted. Also, the “execution data editing” subroutine in step # 250 is replaced with the “execution data editing” subroutine in step # 250R, and the “execution and result display” subroutine in step # 270 is replaced with “execution and result display” in step # 270R. It has been replaced by a subroutine.

  As shown in FIG. 15, in the operation procedure input subroutine # 300, first, in step S302, a combination table of the original state of the state transition table and the operation key is created by a commercially available spreadsheet application. The state transition table represents the test items and the order determined based on the priority for realizing the function of the developed product when the regression test is performed. Then, the control proceeds to the next Step S304.

  In step S304, the contents of the combination table created by the spreadsheet application are converted by the user into a file having a normal database structure such as CSV format. Then, this file is taken into the “combination table”. Then, the control proceeds to the next Step S306.

  In step S306, an operation procedure for creating each original state is defined. For example, when the PLAY key is pressed after the power is turned off, PLAY is defined. Similarly, it is defined that when PLAY is pressed after the power is turned off and then PAUSE is pressed, PAUSE is set. Then, the control proceeds to the next Step S308.

  In step S308, a “command to be sent to the target” corresponding to each operation key is defined. For example, the PLAY key inputs that the RAM flag named F_PLAY is set to 1. Then, the processing in this subroutine ends.

  Then, after the “state name definition” subroutine in step # 100, the “target state data generation” subroutine in step # 400 is started.

  As shown in FIG. 16, in the target state data creation subroutine # 400, first, in step S402, the designated original state is created. For example, a power-off state is created. Then, the control proceeds to the next Step S404.

  In step S404, key-corresponding microcomputer RAM memory data corresponding to the designated target key icon is transmitted to the target microcomputer. Then, the control proceeds to the next Step S406.

  In step S406, a command for reading the corresponding memory data in the microcomputer RAM is transmitted. In response to the command, the response data returned is received and stored in the RAM of the personal computer, and converted into the target state name, and the state name is entered in the table. Then, the control proceeds to the next Step S408.

  In step S408, it is determined whether the combination execution has been completed. If no, control proceeds to step S410.

  In step S410, the next combination is designated. And control returns to above-mentioned step S402.

  On the other hand, if it is determined in step S408 that the combination execution has been completed, the processing in this subroutine ends, and the control proceeds to the next “execution data editing” subroutine # 250R.

  As shown in FIG. 16, in step # 250R, in step # 250 shown in FIG. 6, the “display key name and obtained state name” process in step S26 is replaced with the “result screen display” process in step S26R. In step S29, the “edit contents in the obtained state” process in step S29 is replaced with the “edit table contents” process in step S29R. In addition, the advance of control when it is determined No in step S30 has been replaced with the newly registered “execution?” Process in step S31 from the “edit?” Process in step S28. In addition, a “save with file name” process in step S33 is added as a destination of control when it is determined No in step S31.

  When the process of step # 250R is started, first, in step S26R, the target state data generated in step # 400 is displayed on the screen in the form of a table. Then, the control proceeds to the next S28.

  In step S28, when it is determined not to edit, the control proceeds to step S29R.

  In step S29R, the contents of the target state data table are edited. And control returns to above-mentioned step S26R.

  On the other hand, if it is determined in step S28 that editing is to be performed, control proceeds to step S30.

  In step S31, it is determined whether or not to register the target state data displayed on the screen as an execution result. In No, control returns to above-mentioned step S26R. On the other hand, when it is determined Yes in this step, the control proceeds to step S33.

  In step S33, the current target state data is saved with a file name. And control returns to above-mentioned step S26R.

  Then, in the case where the determination in step S30 is Yes, the processing of the same content as in step # 250 described above is performed, the processing in this subroutine is terminated, and the next “execution and result display” subroutine # 270R starts.

  As shown in FIG. 17, in step # 270R, in step # 270 shown in FIG. 7, between step S42 and step S46, the “sequential execution?” Process in step S43, and “output to spreadsheet application” in step S49. ? "Processing and" application file designation? "Processing in step S51 are added. Further, between step S42 and step S48, a “random designation” process in step S45, an “order designation” process in step S47, and a “target state data generation” subroutine in step # 400 are inserted. Furthermore, the “comparison of status name obtained for each target key and status name in expected value file” in step S50 is replaced with “comparison of status name of execution result table and expected value file table” processing. ing.

  When step # 270R starts, it is first determined in step S42 whether or not random execution is performed. In No, control progresses to step S43.

  In step S43, it is determined whether or not the sequential execution is performed. If no, the control proceeds to step S44.

  In step S44, it is determined whether or not an expected value file is designated. If No, the control proceeds to the next step S49.

  In step S49, it is determined whether to output data to the spreadsheet application. In the case of Yes, the control proceeds to the next step S51.

  In step S51, after the data is output to the spreadsheet application, control returns to step S42 described above.

  On the other hand, when it is determined No in step S49, the control returns to step S42 described above.

  Furthermore, when it is determined Yes in step S44, the control returns to step S42 through step S46 described above.

  If it is determined as Yes in step S43, the control proceeds to step S47.

  In step S47, the order of sequential execution is specified. Then, the control proceeds to step # 400. And when it is judged as Yes at Step S48, control progresses to Step S53.

  In the case of Yes in step S42, after the random designation is made in step S45, the control proceeds to step # 400.

  In step S53, the status names of the execution result table and the expected value file table are compared. Then, the control proceeds to step S55.

  In step S55, the content (shown in FIG. 20) with the comparison result added is displayed in the execution result table (the content shown in FIG. 19 showing the obtained state names). Then, this subroutine ends.

  FIG. 18 shows an example of the “original state / operation key combination table” created in step S302.

  FIG. 19 shows an example of the “result screen” displayed in step S26R.

  FIG. 20 shows an example of the “execution result screen displayed with the comparison result added” displayed in step S55.

  It can be used for software upgrades, software development, etc. when developing a new product that can realize part or most of its functions in software.

1 is a block diagram showing the configuration of an automatic software verification system according to a first embodiment of the present invention. The block diagram which shows the structure of the inspection item setting device shown in FIG. The flowchart which shows the main operation | movement of the software automatic verification system shown in FIG. The flowchart which shows the detailed operation | movement in the state name definition subroutine of step # 100 shown in FIG. A flowchart showing detailed operations in the operation order setting subroutine of step # 150 and the execution data generation subroutine of step # 170 shown in FIG. A flowchart showing detailed operations in the state-corresponding memory data generation subroutine of step # 200 and the execution data editing subroutine of step # 250 shown in FIG. Flowchart showing detailed operations in the execution of step # 270 and the result display subroutine shown in FIG. The schematic diagram which shows the structure of the state data produced | generated by the inspection item setting device shown in FIG. The schematic diagram which shows an example of the operation memory menu displayed by the display shown in FIG. The schematic diagram which shows an example of the execution menu displayed by the indicator shown in FIG. The schematic diagram which shows an example of the edit menu displayed by the indicator shown in FIG. The schematic diagram which shows an example of the comparison result screen displayed by the indicator shown in FIG. The block diagram which shows the structure of the test | inspection item setting device concerning the 2nd Embodiment of this invention. The flowchart which shows the main operation | movement of the test | inspection item setting device shown in FIG. A flowchart showing detailed operations in the state name definition subroutine of step # 300 shown in FIG. FIG. 14 is a flowchart showing detailed operations in the target state data generation subroutine of Step # 400 and the execution data editing subroutine of Step # 250R shown in FIG. FIG. 14 is a flowchart showing detailed operations in the execution of step # 270R and the result display subroutine shown in FIG. Schematic diagram showing an example of the “original state and operation key combination table” displayed by the display shown in FIG. Schematic diagram showing an example of a “result screen” displayed by the display shown in FIG. Schematic diagram showing an example of a “comparison result screen” displayed by the display shown in FIG.

Explanation of symbols

Svs1, Svs2 Software automatic verification system 100A, 100B Inspection item setting unit 101 Display unit 103 Operation unit 105 Operation order setting unit 106 Operation order storage unit 107 Expected value storage unit 109 Comparator 111 Command transmitter 113 Data receiver 115 Execution result storage Device 117 state-corresponding memory storage device 119 target state name converter 120 state name conversion table manual creation / registration device 132 input device from spreadsheet application 134 output device to spreadsheet application 200 verification target device 201 command receiver 203 Controller 205 Device control memory 207 Key correspondence memory writer 209 State correspondence memory reader 211 Data receiver

Claims (10)

A software automatic verification method that supports the creation of a software regression test method that is created when developing software for use in products whose functions are implemented in software,
Defining the state of each function of the product;
Associating the defined function with a memory data RAM name of a microcomputer that controls the function of the product;
Converting the associated function into an absolute address of memory data;
Performing a regression test on the product;
A method of automatically verifying software, comprising: inversely converting memory data at an absolute address representing a test result obtained after the regression test into a function. Determining the content and order of regression testing for each of the associated functions;
The software automatic verification method according to claim 1, further comprising a step of executing a regression test on the product in a determined order.   The software automatic verification method according to claim 2, further comprising a step of expressing a state name of the inversely converted function.   The software automatic verification method according to claim 2, further comprising the step of storing the determined test order. A function state name expected as a test result is set as an expected value, and the content is compared with the content of the state name of the inversely converted function;
The software automatic verification method according to claim 3, further comprising a step of displaying the comparison result.   Create a combination of inspection items in advance with a predetermined software tool to obtain the results for all the inputs that can be in the original state, with the state of the function considered as the basic function of the product as the original state. 6. The software automatic verification method according to claim 5, wherein the software can be used.   The software automatic verification method according to claim 3, further comprising a step of editing the content and order of at least one of the expected value and the regression test using a user judgment on the test result as an input. The software automatic verification method according to claim 7, further comprising a step of determining the editing result as the expected value.   A computer program capable of executing the automatic verification method according to claim 1 by being read by the computer. A computer program product comprising a computer-readable medium provided with computer code means capable of executing the automatic verification method according to claim 1 by being read by the computer.

Images