AU2013228057A1 - Software System for Breaking Online Benchmarking Methodologies into Discrete Tasks - Google Patents

Abstract

Abstract The present invention is a system, method and computer software program that automates the process of benchmarking websites or other software. This type of benchmarking involves addressing how to: select the right research questions; ensure questions are asked in context; ensure that early research findings modify later research questions if necessary; and compile the research answers into a report or file containing charts, graphs, screenshot images, tables and text. These four issues are so difficult that typically a business must engage human experts to manage their benchmarking. The invention is a novel computer software program that removes the need for this manual research management by automatically selecting and tailoring questions, modifying them for context, sending them to a range of human and/or non-human testers that work in parallel, and collating the results into a report or file containing charts, graphs, screenshot images, tables and text. 'a ~ o~ '~ 9 0u

Description

1 Australia Patents Act 1990 Patent Specification FOR A STANDARD PATENT [0001] This application claims priority based on the provisional patent application entitled "Software System for Breaking Online Benchmarking Methodologies into Discrete Tasks," filed September 17, 2012 Application number 2012904023. [0002] The following statement is a full description of this invention, including the best method of performing it known to me: Field of the Invention [0003] The present invention generally relates to the field of computer software. The present invention also relates to a method and system of benchmarking the content, features and functions provided by websites, mobile phone applications, tablet applications and other forms of software (Online Benchmarking) using an online, web-based computer software program. Background [0004] Benchmarking is the process of comparing a business's processes and performance to industry best practices. It involves identifying the best performers in an industry or another similar industry (the targets) and comparing the results and processes of the targets to a business's own results and processes. In this way a business learns how well the targets perform and importantly the factors that explain why the targets are successful. [0005] There is no universally accepted benchmarking process. However, there are a number of existing methodologies. As benchmarking can be applied to any 2 business process or function, a range of research techniques may be required. They can include informal conversations with customers, employees or suppliers; exploratory research techniques such as focus groups; or in-depth market research, quantitative research, surveys, questionnaires, re-engineering analysis process mapping, quality control variance reports, and financial ratio analysis. [0006] Online Benchmarking is the process of grading various aspects of the content, features and functions provided by websites, mobile phone applications, tablet applications and other forms of software. Successful Online Benchmarking involves addressing four difficult issues. [0007] The first difficult issue that must be addressed is how to select the right research questions. For example, when testing an online retailer's website you would typically ask whether the retailer offered Alipay and cash-on delivery only if the retailer was selling into China. Likewise, you would only research the quality of its sizing guide if it was selling apparel, footwear or accessories. Drilling down to ask the specific questions that would provide meaningful data for a business can be time consuming and difficult. Without the invention described in this patent, resolving this difficult issue requires manual intervention from a human expert. [0008] The second difficult issue that must be addressed is how to ensure questions are asked in context. For example, a research question might ask whether there are useful product images on an online retailer's website. In the context of benchmarking online bookshops, answering this question might involve checking for a front and back cover image and an image of the table of contents. In the context of benchmarking online appliance retailers, answering this question might involve checking for front and back product photos as well as close-ups of all sockets and controls. Without the invention described in this patent, resolving this difficult issue requires manual intervention from a human expert. [0009] The third difficult issue that must be addressed is how to ensure that early research findings modify later research questions if necessary. For example, if a question asked about the quality of website's sign-up form, then it might only make 3 sense if the answer to an earlier question - whether the website actually has a sign-up form - was affirmative. Without the invention described in this patent, resolving this difficult issue requires manual intervention from a human expert. [0010] The fourth difficult issue is compiling the research answers into a report or file containing charts, graphs, screenshot images, tables and text. Without the invention described in this patent, resolving this difficult issue requires manual intervention from a human expert. [0011] These four issues are so difficult that a business must engage human experts to manage their Online Benchmarking. These experts tailor the research criteria for each benchmarking project, following a manual process that involves lengthy negotiations with the customer as well as reviewing previous projects to find research questions that can be re-used. The analyst then typically coaches a research team on how to apply the benchmarking criteria and writes a report based on the research context and findings as well as the customer's needs. [0012] This manual process is complex, time consuming and expensive. A research team might spend months completing a research project for a customer, especially if it is a small team and its members have other commitments. These costs and delays make Online Benchmarking unaffordable and inaccessible to many businesses, in particular small to medium sized businesses. In addition, the delay in providing the report in a rapidly changing global market may render the results obsolete before the report is complete. This means many businesses are unable to afford Online Benchmarking. In the modern commercial world where websites and apps are vital business tools, being able to regularly engage in Online Benchmarking to increase competitiveness and profitability is integral to the viability of many businesses. [0013] This invention is novel as Online Benchmarking is provided by a computer software program that automatically selects and tailors research questions, puts them in context, ensures they are answered, handles any dependencies between 4 questions, and generates the final report or file, including all charts, graphs, screenshot images, tables and text. [0014] The invention is also novel as the computer software co-ordinates the efforts of an unlimited number of testers (Testers), which can be any combination of human testers and/or non-human testers, which are testers in the form of software. The ease of access and cost reduction provided by this invention make the activity of Online Benchmarking more accessible to a variety of businesses and will thus increase the viability of businesses in the domestic and global economy. In this way the invention is also useful. Summary [0015] In the domestic and global marketplaces where the internet and in particular websites and apps are used increasingly for trading, having access to an efficient and cost-effective Online Benchmarking system is integral for all businesses. [0016] Online Benchmarking experts are skilled in the art of Online Benchmarking. Online Benchmarking experts consider it obvious that human experts must manually select the right questions for each benchmarking project, tailor those questions to the client's need and research context, coach any members of the research team who are less experienced, manage the dependencies that exist between various research questions, and generate the resulting report or file including charts, graphs, tables, images and text. It is not obvious to Online Benchmarking experts that an invention like the one described in this patent application can automate any of these tasks. Rather, Online Benchmarking experts consider Online Benchmarking to necessarily involve expert manual human intervention. This invention is new and innovative as it interrupts what is commonly taken to require expert manual human intervention by automating Online Benchmarking projects and breaking them into discrete tasks that are performed by any number of Testers. In this way a business is able to access Online Benchmarking by purchasing a product rather than engaging the services of an Online Benchmarking expert. This means a business can access Online Benchmarking quickly and efficiently.

5 [0017] In accordance with the present invention a business is able to order its Online Benchmarking report directly through a website. A business is able to answer an online questionnaire about its situation and Online Benchmarking requirements so that the computer software program can select the right research questions to use for each Online Benchmarking project from a database of potential research questions. Each research question is a discrete research activity for the computer software program. The question is answered by a discrete online tester, which may be a human tester or a non-human tester in the form of software ("the Tester"). [0018] To ensure a high-quality response is obtained from a Tester, the computer software program tailors each question if necessary and sends detailed instructions for each question. The Testers work in parallel to answer each discrete research question. As such, the total time to build a Online Benchmarking report is just a few multiples of the longest time taken to answer one research question. [0019] The computer software program collates the findings of the Testers and automatically draws graphs and charts, generates tables and text, captures screenshot images, and compiles these components into reports and files for the Customer. [0020] For example, if an Online Benchmarking project involves the computer software program obtaining answers to 100 research questions about four websites (the Customer's and three operated by competitors) it is possible for the report to be generated in less than a day. The research performed by this invention is provided at a significantly reduced cost and many times faster than by current manual Online Benchmarking methods, which can take months. This allows Online Benchmarking as a business method to be more accessible to a wide range of businesses worldwide. The use of a computer software program to provide Online Benchmarking in this way is new and innovative.

6 Brief Description of the Drawing [0021] Embodiments of the present invention are hereinafter described with reference to the accompanying flowchart titled Figure 1 that shows a diagram of the process of the Online Benchmarking method and the function of the computer software program for the purpose of illustration. Detailed Description of the Invention [0022] A computer program that provides a method and system for benchmarking the content, features and functions provided by websites, mobile phone applications, tablet applications and other forms of software (Online Benchmarking) using an online, web-based computer software program is disclosed. In the following detailed description of the embodiments of the invention reference is made to the accompanying flowchart that forms a part hereof and is titled Figure 1 and in which are shown by way of illustration specific embodiments in which the invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention and it is understood that other embodiments may be utilised and that changes may be made without departing from the scope of the present invention. The following detailed description is therefore not to be taken in a limiting sense and describes only the present invention. Modifications and/or changes can be made thereto without departing from the scope and spirit of the invention, the embodiments being illustrative and not restrictive. It will however be apparent to one of ordinary skill in the art that the invention may be practiced without these specific details. In other instances well-known features have not been described in detail to avoid unnecessarily complicating the description. [0023] In this section: (a) the reference to a step refers to a numbered step in the flow chart titled Figure 1; and 7 (b) System means the invention which is comprised of a computer software program that automates Online Benchmarking projects. [0024] In one or more of the embodiments, the System includes an online payment system that allows a business customer ("the Customer") to order an Online Benchmarking report. [0025] In one of more of the embodiments, the System includes a research questionnaire or survey. At Step [1] the business customer ("the Customer") who wishes to purchase an Online Benchmarking report accesses the System by logging into the System using a web browser or another online application. The Customer is prompted by the System to enter their research needs or parameters. For example, the System asks the Customer which websites or other software it should test. The Customer might want the System to test the Customer's own website, along with some websites operated by its closest competitors and/or any other companies that the Customer considers to be relevant. The System asks the Customer which of the System's research products it should use to test the selected websites. [0026] Many of the System's available research products are designed to test a particular aspect of a particular class of website or other software. For example, a Retail Site Search Benchmark would be a research product designed to test the quality of the search experience on retail websites. A Media Social Media Benchmark would be a research product designed to test the quality of the social media experience on media websites. The system contains a variety of research products, which may or may not include a Retail Site Search Benchmark and a Media Social Media Benchmark. The System asks the Customer to choose from the available research products. The System may prompt the Customer to enter additional research needs. [0027] In one or more of the embodiments the System is a database of research questions. At Step [2] the System loads the research questions that sit within the research product that the Customer selected. The research product that the Customer selected contains a range of research questions. Some of the research 8 questions might be relevant when testing certain websites and apps but not others. For example, a research question about whether a website accepts customer payments via Alipay would be very relevant when testing an online retailer that has customers in China, but not relevant when testing an online retailer that only had customers in the United States. In Step [2], the System selects the research questions that are relevant to the research activity given the needs that the Customer entered in Step [1]. [0028] In one or more embodiments the System analyses the research questions selected by the Customer and modifies as required. At Step [3], the System modifies some of the selected questions if necessary, depending on the Customer's research needs and the context of the research. For example, consider a research question that looks for useful product images on an online retailer's website. This research question might have to be modified depending on the context of the research and the parameters initially entered by the Customer. When checking an online bookshop, this question might have to check for a front and back cover image and an image of the table of contents. When checking an appliance retailer, it might have to check for front and back product photos as well as close-ups of all sockets and controls. In Step [3], the System modifies each of the selected research questions if necessary given the research context and the needs the Customer entered in Step [1]. [0029] At Step [4] the System selects the first research question from the set of questions selected for the Customer. [0030] In one or more embodiments the System organises the order in which research questions are asked. At Step [5] the System determines whether another research question is required to be answered before the selected question can be answered as some questions might depend upon the answers to other questions. For example, if the selected question asked about the quality of the website's sign up form, then this question might only make sense once the System has answered another question about whether the website actually has a sign-up form. In Step [5], the System checks whether the selected question depends on another question. If 9 the selected question does not depend on another question, the System moves to Step [6]. [0031] If the selected question does depend on another question, the System moves to Step [10]. [0032] In one or more embodiments the System conducts research. At Step [6] if the research question does not rely on the answer to another question then the System sends the research question with the relevant instructions to the Tester for answering. This Tester can be an employee of the entity that owns the System, or a human tester that is not directly employed by the entity, or a module within the System itself that answers the question automatically, or a software service outside the System that answers the question automatically. [0033] At Step [7] the System gathers and stores the answer received from the Tester. [0034] At Step [8], the System checks if all the questions have been answered. If they have, the System moves to Step 14. If they have not, it moves to Step [9]. [0035] At Step [9], the System selects the next unanswered question from the set of research questions. The system then returns to Step [5]. [0036] At Step [10], the System checks whether the other question that the selected question depends upon has been answered. If this other question has been answered, the System moves to Step [11]. If this other question has not been answered, the System moves to Step [9], in which it selects a new question for processing. [0037] At Step [11] if the preliminary question has been already answered the System reviews the response and determines whether the research question at Step [4] can be answered. For example, consider a selected question that asks about the quality of website's sign-up form, and that depends on another question about 10 whether the website actually has a sign-up form. If the other question found that the website did have a sign-up form, the selected question could now be asked or answered, and when Step [11] finds the selected question can now be asked and answered, the System moves to Step [13]. On the other hand, if the other question found that the site did not have a sign-up form, then the selected question should be removed from the set of research questions, and when Step [11] finds the selected question can now be asked or answered, the System moves to Step [12]. [0038] At Step [12], the System removes the selected question from the set of research questions, because Step [11] found that the selected question cannot be asked and answered, because of the answer to another question on which the selected question depends. The System then moves to Step 8. [0039] In Step [13], the System modifies the selected question if required. The System reaches Step [13] because Step [11] found that the selected question can be asked and answered, because of the answer to another question on which the selected question depends. It is possible that the selected question has to be modified in light of the answer to the other question on which it depends. For example, consider a selected question that checks whether a "Search" button worked efficiently. Consider the case where it depends on another question, which asked what this button was called on the website that the System is testing. If the answer to that other question was "Find" not "Search", then the selected question might be modified to ask whether the "Find" button worked efficiently. In Step [13], the System modifies the selected question if required. The system then moves to Step [6]. [0040] In one or more of the embodiments the System prepares an Online Benchmarking report. At Step [14], the System uses the research answers, along with the parameters that the System originally collected from the Customer and the context of the research, to automatically build a report or file for the Customer. The System can now does this, because when the System reaches Step [14], all the research questions have been either answered or removed from the set of selected questions.

11 [0041] The System uses the available information to create a report or file that includes: the websites or apps that were tested, the research questions that were used to test those websites or apps, how well each website or app performed on each research question, the reason each website or app was judged to be performing at that level, each website or app's overall performance against all questions, and each website or app's overall performance against each significant group of questions. The report or file generated by the System is available to download from the site. Industrial Applicability [0042] It is apparent from the above that the arrangements described are applicable to the industries of computer and data processing, business systems and processes, and any business that uses a website for any purpose, including but limited to marketing and retail or wholesale trade.

Claims (12)

1. A computer program that allows a user to order from a website a report that benchmarks the content, features and functions provided by websites, mobile phone applications, tablet applications and other forms of software.

2. A computer program according to claim 1 wherein a user sets the parameters of their benchmarking project by providing a database of research questions that can be modified and tailored to the specific business of the user.

3. A computer program according to claim 1 wherein a benchmarking report is generated by a computer software program that interfaces with an infinite variety of Testers that respond rapidly to research questions.

4. A computer program according to claim 1 wherein a research question is asked and an answer to the research question is processed.

5. A computer program that sells, manages and delivers benchmarking research as a product.

6. A computer program according to claim 5 wherein a user can order, purchase and receive an electronic benchmarking report online.

7. A computer program according to claim 3 wherein research can be undertaken by any number of Testers working in parallel.

8. A computer program that systematically organises the order in which research questions selected by the user are sent to Testers.

9. A computer program that processes and stores the answers to research questions as they are received from Testers.

10. A computer program that collates and analyses the answers to research questions provided from Testers. 2

11. A computer program according to claim 5 that generates a benchmarking report for the user based on the answers provided by the Testers.

12. A computer program according to claim 11 wherein a benchmarking report is stored and can be downloaded by the user.