EP1716483A2 - Systemes et procedes pour le developpement de logiciel - Google Patents

Systemes et procedes pour le developpement de logiciel

Info

Publication number
EP1716483A2
EP1716483A2 EP05705832A EP05705832A EP1716483A2 EP 1716483 A2 EP1716483 A2 EP 1716483A2 EP 05705832 A EP05705832 A EP 05705832A EP 05705832 A EP05705832 A EP 05705832A EP 1716483 A2 EP1716483 A2 EP 1716483A2
Authority
EP
European Patent Office
Prior art keywords
software
developers
design
development
program
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
EP05705832A
Other languages
German (de)
English (en)
Other versions
EP1716483A4 (fr
Inventor
John M. Hughes
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Topcoder LLC
Original Assignee
Topcoder LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Topcoder LLC filed Critical Topcoder LLC
Priority to EP11001173A priority Critical patent/EP2333657A1/fr
Publication of EP1716483A2 publication Critical patent/EP1716483A2/fr
Publication of EP1716483A4 publication Critical patent/EP1716483A4/fr
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00—Arrangements for software engineering
    • G06F8/20—Software design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06Q—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q10/00—Administration; Management

Definitions

  • This invention relates to computer-based methods and systems for developing and distributing software and, more particularly, to methods and systems facilitating the distributed development of software.
  • the invention relates to providing infrastructure, process controls, and manpower to develop software using a repeatable, structured model in order to transform software development from an ad-hoc, custom development exercise into a streamlined, predictable manufacturing operation.
  • this goal can be achieved, in one exemplary implementation, by separating the software design functions from the software development functions, providing rigorous review processes, and using a competition model whereby a number of distributed, unrelated, and motivated design or program is selected.
  • software development firms can be employed to perform only a portion of an entire process. For example, a consulting firm may be hired to develop a functional specification for an application addressing a certain business need, or an offshore programming shop engaged to build software according to the specification.
  • a multi-step software development manufacturing process that has well-defined inputs and outputs at each step, meets stringent quality control requirements, and catalogs each process output as a subassembly component of the larger product, allows such flexibility without sacrificing quality.
  • Such a process can be entered (or exited) at various points, while the independence of the developers allows for enforcement of rigorous design and quality analysis without "office politics" or other favoritism, which in turn results in very high quality (e.g., enterprise quality) software.
  • a specification for the design of a software program such as a component, an application, a module, or a library is communicated to a first plurality of developers, who in some cases may be geographically distributed.
  • designs which may or may not include such items as a requirements document, an activity diagram, a case document, test cases, a prototype, or a UML document for the software program are received from a subset of the plurality of developers.
  • a design review process for review of the received designs is facilitated, and based at least in part on the results, one design is selected.
  • the selected design is communicated to a second plurality of software developers (who in some cases may be different than the first plurality of software developers), and in response to the communicated design, a software program, which in some cases may comprise source code, object code, or compiled code, is received from each of the second plurality of software developers.
  • a software review process is facilitated for reviewing each of the received programs, and one program is selected based at least in part on its review.
  • Various embodiments can include one or more of the following features. Developers' skill ratings can be derived from the developer's performances in one or more coding competitions, which (in whole or in part) can be held online. For example, the first plurality of software developers can be selected based, at least in part, on having achieved a minimum rating received in the one or more competitions. The ratings assigned to a developer can be derived (in whole or in part) from the score associated with one or more designs or programs.
  • a difficulty level can be associated with the software program, and in some embodiments, the developers can be rated based, at least in part, on the difficulty level associated with the design or program.
  • a portion of the specification can be received from an entity requesting the development of a software program.
  • the specification can be communicated using an on-line application, using, for example, the Internet.
  • the method can further include rewarding the software developer that submitted the selected design or program with, for example, monetary rewards and/or increased skill ratings.
  • the design review and/or software review processes can be performed by a plurality of reviewers, who in some cases may have been previously rated in a computer programming competition, and may have achieved a rating above a predetermined minimum rating.
  • the design and/or software review process can include aggregating scores from each of the plurality of reviewers into a summary rating, and the selection of one design or program can be based on the summary score.
  • the design review process can include one or more activities such as reading design documents, completing a review form, which in some cases my be an on-line form, and identifying changes to be incorporated into the design by the software developer who submitted the design. The changes can be designated as mandatory or optional at the discretion of the reviewer.
  • an appeal can be made contesting the score assigned to a software developer's design and/or program.
  • a selected program can be distributed, and in some cases support for the distributed program may be provided.
  • Another aspect of the invention relates to a method of distributed software development.
  • the method includes providing a software development system to a distributed community of software developers, accepting a request to create a software program from an entity, and facilitating the development of the software program by at least a subset of the community of software developers using the software development system.
  • the software development system provided to the developers includes software development software that facilitates the development and testing of software programs using a structured development methodology, which in some cases may include multiple phases such as a specification phase, a design phase, a development phase, a testing phase, and a support phase; a communication server in communication with the software development software for delivering the software development software to a distributed community of software developers; and a review board comprising one or more programmers capable of determining the quality of software developed by the distributed community of software developers.
  • the software programs can be programs selected from the group of components, applications, modules and libraries.
  • the software developers can be rated based on their participation in one or more coding competitions or the designs and code that they develop.
  • one or more components of the software development system can be geographically distributed, and the distribution can be effected using the Internet or other networks.
  • the client software used by programmers to develop computer code can be in the form of a downloadable applet (i.e., a Java applet).
  • the review board can include programmers that were previously rated, for example, in a coding competition or by rating the designs and/or code that they have developed.
  • the quality of the software can be determined by the review board by reviewing such items as design documents, source code, object code, compiled code, class definitions, and methods.
  • the development environment can facilitate the development of such items as design models (e.g., UML models), case models, and computer code, as well as the compilation and testing of computer code.
  • the method can also include receiving developed software programs from a subset of the community of software developers, and in some cases selecting one or more of the received programs to be delivered to the entity.
  • the developers that submitted the one or more selected software programs may be compensated, in some cases with money, and in some cases by an increased skill rating.
  • the invention provides a computerized method for evaluating software programs.
  • the method includes communicating requirements for the development of a software program to a population of software developers, and in response, receiving from each of a subset of the population of software developers a candidate software program and one or more test cases for testing the received candidate software program.
  • the method also includes testing each of the received software programs using test cases received from two or more of the subset of the population of software developers, and scoring the received candidate software programs based at least in part on the results of the testing.
  • Embodiments can include one or more of the following features.
  • the software developers can be geographically distributed, and in some cases may have been rated in one or more coding competitions.
  • the candidate software programs can include source code, object code, compiled code, class definitions, methods, applications, and components.
  • the submitted test cases can include sample data to be used as input for the candidate software program.
  • the invention relates to systems for implementing the methods just described.
  • a system for evaluating the functionality of software programs includes a communications server for communicating requirements for the development of a software program to a population of software developers, and in response, receiving from each of a subset of the developers a candidate software program and one or more test cases for testing the received program, a testing server for testing each of the received software programs using test cases received from two or more of the subset of software developers, and a scoring server in communication with the testing server for scoring the received candidate software programs based at least in part on test results received from the testing server.
  • the software developers can be geographically distributed, and in some cases the developers may have been previously rated.
  • the received candidate software programs can include source code, object code, compiled code, class definitions, methods, applications, or components, and the submitted test cases can include sample data to be used as input for the candidate software program.
  • FIG. 1 is a block diagram of an embodiment of a distributed software development system having a server according to the invention.
  • FIG. 2 is a block diagram of one embodiment of a software development domain according to an embodiment of the invention.
  • FIG. 3 is a flow chart depicting steps performed in developing a software program according to an embodiment of the invention.
  • FIG. 4 is a flow chart depicting an overview of the operation of an embodiment of the invention.
  • FIG. 5 is a block diagram depicting a software testing environment created with multiple submissions of test cases according to an embodiment of the invention.
  • FIG. 6 is a more detailed diagram of an embodiment of a testing environment such as that shown in FIG. 5.
  • FIG. 7 is a block diagram of an embodiment of a server such as that of FIG. 1 to facilitate the development and/or testing of software programs.
  • a distributed software development system 101 includes at least one server 104, and at least one client 108, 108', 108", generally 108.
  • the distributed software development system includes three clients 108, 108', 108", but this is only for exemplary purposes, and it is intended that there can be any number of clients 108.
  • the client 108 is preferably implemented as software running on a personal computer (e.g., a PC with an INTEL processor or an APPLE MACINTOSH) capable of running such operating systems as the MICROSOFT WINDOWS family of operating systems from Microsoft Corporation of Redmond, Washington, the MACINTOSH operating system from Apple
  • the client 108 could also be implemented on such hardware as a smart or dumb terminal, network computer, wireless device, wireless telephone, information appliance, workstation, minicomputer, mainframe computer, or other computing device, that is operated as a general purpose computer, or a special purpose hardware device used solely for serving as a client 108 in the distributed software development system.
  • clients 108 can be operated and used by software developers to participate in various software development activities. Examples of software development activities include, but are not limited to software development projects, software design projects, testing software programs, creating and/or editing documentation, participating in programming contests, as well as others. Clients 108 can also be operated by entities who have requested that the software developers develop software (e.g., customers). The customers may use the clients 108 to review software developed by the software developers, post specifications for the development of software programs, test software modules, view information about the developers, as well as other activities described herein. The clients 108 may also be operated by a facilitator, acting as an intermediary between the customers and the software developers.
  • a facilitator acting as an intermediary between the customers and the software developers.
  • the client computer 108 includes a web browser 116, client software 120, or both.
  • the web browser 116 allows the client 108 to request a web page or other downloadable program, applet, or document (e.g., from the server 104) with a web page request.
  • a web page is a data file that includes computer executable or interpretable information, graphics, sound, text, and/or video, that can be displayed, executed, played, processed, streamed, and/or stored and that can contain links, or pointers, to other web pages.
  • a user of the client 108 manually requests a web page from the server 104.
  • the client 108 automatically makes requests with the web browser 116.
  • Examples of commercially available web browser software 116 are INTERNET EXPLORER, offered by Microsoft Corporation, NETSCAPE NAVIGATOR, offered by AOL/Time Warner, or FIREFOX offered the Mozilla Foundation.
  • the client 108 also includes client software 120.
  • the client software 120 provides functionality to the client 108 that allows a software developer to participate, supervise, facilitate, or observe software development activities described above.
  • the client software 120 may be implemented in various forms, for example, it may be in the form of a Java applet that is downloaded to the client 108 and runs in conjunction with the web browser 116, or the client software 120 may be in the form of a standalone application, implemented in a multi-platform language such as Java or in native processor executable code.
  • the client software 120 if executing on the client 108, the client software 120 opens a network connection to the server 104 over the communications network 112 and communicates via that connection to the server 104.
  • the client software 120 and the web browser 116 may be part of a single client-server interface 124; for example, the client software can be implemented as a "plug-in" to the web browser 116.
  • a communications network 112 connects the client 108 with the server 104. The communication may take place via any media such as standard telephone lines, LAN or WAN links (e.g., Tl, T3, 56kb, X.25), broadband connections (ISDN, Frame Relay, ATM), wireless links (802.11, bluetooth, etc.), and so on.
  • the network 112 can carry TCP/IP protocol communications, and HTTP/HTTPS requests made by the web browser 116 and the connection between the client software 120 and the server 104 can be communicated over such TCP/IP networks.
  • the type of network is not a limitation, however, and any suitable network may be used.
  • Non-limiting examples of networks that can serve as or be part of the communications network 112 include a wireless or wired ethernet-based intranet, a local or wide-area network (LAN or WAN), and or the global communications network known as the Internet, which may accommodate many different communications media and protocols.
  • the servers 104 interact with clients 108.
  • the server 104 is preferably implemented on one or more server class computers that have sufficient memory, data storage, and processing power and that run a server class operating system (e.g., SUN Solaris, GNU/Linux, and the MICROSOFT WINDOWS family of operating systems).
  • a server class operating system e.g., SUN Solaris, GNU/Linux, and the MICROSOFT WINDOWS family of operating systems.
  • Other types of system hardware and software than that described herein may also be used, depending on the capacity of the device and the number of users and the size of the user base.
  • the server 104 may be or may be part of part of a logical group of one or more servers such as a server farm or server network.
  • the server 104 also can include a contest server, such as described in U.S. Patent Nos. 6,569,012 and 6,761,631, entitled “Systems and Methods for Coding Competitions” and “Apparatus and System for Facilitating Online Coding Competitions” respectively, both by Lydon et al, and incorporated by reference in their entirety herein.
  • the server 104 and clients 108 enable the distributed software development of a software program by one or more developers, which developers may or may not be associated with the entity requesting the development of the software program.
  • the software program can be any sort of instructions for a machine, including, for example, without limitation, a component, a class, a library, an application, an applet, a script, a logic table, a data block, or any combination or collection of one or more of any one or more of these.
  • the software program is a software component.
  • a software component is a functional software module that may be a reusable building block of an application.
  • a component can have any function or functionality.
  • software components may include, but are not limited to, such components as graphical user interface tools, a small interest calculator, an interface to a database manager, calculations for actuarial tables, a DNA search function, an interface to a manufacturing numerical control machine for the purpose of machining manufactured parts, a public/private key encryption algorithm, and functions for login and communication with a host application (e.g., insurance adjustment and point of sale (POS) product tracking).
  • components communicate with each other for needed services (e.g., over the communications network 112).
  • a specific example of a component is a JavaBean, which is a component written in the Java programming language.
  • a component can also be written in any other language, including without limitation Visual Basic, C++, Java, and C .
  • the software program is an application.
  • the application may be comprised of one or more software components.
  • the software application is comprised of software components previously developed using the methods described herein.
  • the application comprises entirely new software programs.
  • the application comprises a combination of new software programs and previously developed software programs.
  • a software development domain 204 can be used to provide an entity 208 with high-quality software.
  • One or more developers can be identified and/or selected by various methods from a distributed community of programmers 212, and subsequently used to develop software components.
  • the developers can be employees of, consultants to, or members of an organization, enterprise, or a community fostering collaborative computer programming and distributed software development, and in some cases the developers may have no other formal or informal relationship to each other.
  • one or more of the developers can act as a product manager who is responsible for organizing and coordinating the efforts of other developers.
  • the product manager may also specify items such as, without limitation, the cost of the project, the project schedule, and the project risks.
  • the product manager creates a project plan for the project, which may include, without limitation, an estimated project cost and schedule, and a requirements document describing, for example, the scope and risks of the project.
  • the developers may include architects, designers, programmers, quality assurance engineers, as well as other software development roles as described in co-pending U.S. Patent Application Serial No. 10/408,402, entitled “Method and Systems for Software Development” by Hughes, and incorporated by reference in its entirety herein.
  • the software development domain 204 includes a communication server 216, one or more structured development methodologies 220, software development software 224, and a review board 228.
  • the communication server provides a conduit through which the external entity 208, the community of programmers 212, and the review board 228 can interact, for example, to provide documentation, submit software, elicit and offer feedback, review submitted software, and potentially rate submitted software, either in design or functional form.
  • the communication server is or operates as part of the server 104 as described above, whereas in other cases the communication server may be a separate server, which may be operated by and/or outsourced to an application service provider (ASP), internet service provider (ISP), or other third-party.
  • ASP application service provider
  • ISP internet service provider
  • the structured development methodology 220 provides a framework for the development of software programs.
  • the methodology 220 specifies a common vocabulary, a fixed set of deliverables, development phases or steps, inputs and outputs for one or more of the steps, as well as other aspects of the development process.
  • the methodology 220 bifurcates the development process into an architecture and design phase and a development and testing phase.
  • the outputs of the architecture and design phase such as class diagrams, test cases, technical specifications, and other design documents, are submitted, reviewed, and finalized prior to initiating any development work.
  • the design documents are used as input into the development phase.
  • the developer(s) create source code, scripts, documentation, and other deliverables based on the design documents.
  • the developers are afforded a complete and accurate representation of what it is they are being asked to develop. Furthermore, by using a structured methodology, the participants, (e.g., developers 212, the entity 208) can communicate effectively, and the outputs of each process step are known and can be verified. By providing a common definition, and a known set of inputs, such as use cases, and a known set of outputs such as expected results, and facilitating community-based development, the developers can interact with each other effectively and efficiently, thus reducing the cost and time necessary to produce quality software.
  • the software development software 224 provides an operational mechanism for implementing the methodology 220, and a software development environment in which the developers can do one or more of develop, test, submit, and verify software designs and software programs.
  • components of the software 224 may reside on the server 104, whereas some components may be included in client software residing on a client, e.g., as described above.
  • the software development software 224 optionally can include one or more such as a development library, from which developers can access previously developed components and documentation templates; a documentation feature that provides information about programming terms, syntax, and functions; a compiler that also allows a developer to identify and correct programming errors; and even version control and code management functions.
  • FIG. 3 provides a summary illustration of one embodiment of a method for developing software, for example, using the software development domain 204 described above.
  • the communication server 216 receives a specification (STEP 304) describing the desired functions of a software program, which is then distributed to the distributed community of programmers 212 (STEP 308).
  • One or more of the programmers in the community 212 creates a design detailing the technical aspects of the program based on the functionality described in the specification, and once completed, the design(s) are received at the server 104 (STEP 312).
  • the submitted design(s) are then subject to a design review process (STEP 316) whereby the design(s) are compared to the specification, and evaluated on their implementation of the specified functionality and compliance with the structured development methodology 220.
  • a design that is the "best" of the submissions may be selected in response to the evaluations (STEP 320), and if there is at least one submission of sufficient quality, the selected design may be made available to the distributed community of programmers 212 (STEP 324).
  • Each of a number of programmers submits a software program that they believe conforms to the design and the requirements of the structured development methodology 220.
  • the software programs are received at the server 104 (STEP 328) and the programs are subjected to a software review process (STEP 332) to determine which submitted program(s) best conform to the distributed design and the structured development methodology 220.
  • FIG. 4 provides one possible implementation of the general method described above.
  • the development process is monitored and managed by a facilitator 400.
  • the facilitator 400 can be any individual, group, or entity capable of performing the functions described here.
  • the facilitator 400 can be selected from the distributed community of developers 208 based on, for example, achieving exemplary scores on previously submitted software designs and/or programs, or achieving a high ranking in a software programming contest.
  • the facilitator 400 can be appointed or supplied by the entity (e.g., entity 208) requesting the development of the software program, and thus oversee the design and development process for further assurance that the end product will comport with the specifications.
  • the facilitator 400 receives input from an entity (not shown) wishing to have a software program, application, component, or other asset developed on their behalf.
  • the entity can be a company looking to have one or more computer programs designed and/or developed for internal use, or as portions of larger applications that they intend to sell commercially.
  • the entity provides a detailed specification, and in other cases only a list of functional requirements may be provided.
  • the facilitator receives either the requirements (STEP 406), the specification (STEP 408), or in some cases both from the external entity.
  • the facilitator can develop a specification in accordance with the requirements (STEP 410).
  • one or more members of the development community 407 e.g., development community 212 in FIG. 2 may be asked to develop the specification, and in some cases multiple specifications may be submitted, with one of the submissions selected as the final specification to be used for guiding the design and development efforts.
  • the specification defines the business plan and a stable hardware and/or software platform, or other architectural constraints.
  • the specification can define the network devices, servers, and general infrastructure to support the development and production of the project and product.
  • the specification can also identify a language or tools that the component must be programmed in or with, a functional overview of the software component, boundary conditions, efficiency requirements, computer platform/environment requirements, interface requirements, performance criteria, test-case requirements, and/or documentation requirements of the component.
  • the specification can include an amount of money that will be paid to the designer who submits the best design and/or program that complies with the specification.
  • the specification is assigned a difficulty level, or some similar indication of how difficult the facilitator, entity, or other evaluator of the specification, believes it will be to produce a comprehensive design according to the specification.
  • the difficulty level may, in some cases, also be based on the effort believed to be necessary to complete the task, and the time allotted to complete the task.
  • the difficulty level may be expressed in any suitable manner, for example as a numerical measure (e.g., a scale of 1 to 10), a letter grade, or a descriptive such as easy, medium, or hard.
  • a specification for the design of a complex gene-sequencing algorithm may have a difficulty level of 9 on a scale of 1 to 10, whereas a simple component that performs a search for specific text in a file may be assigned a difficulty level of 2. If there are additional practical constraints, for example if the search component is needed in two days, the difficulty level optionally may be increased due to the tight time constraints. In some embodiments, an award to the designer (e.g., money, skill rating, etc.) that submits the selected design may be produced or adjusted based in part on the difficulty level associated with the specification.
  • the facilitator 400 reviews the specification to determine if it meets the requirements for a complete specification according to the development methodology 220.
  • the methodology can include best-practice activities, templates, guidelines, and standards that assist software architects, programmers, and developers in producing quality code in a consistent and efficient manner. The use of such a methodology reduces the need to rethink and recreate programming documentation and constructs, thus reducing project duration, cost, and increasing quality and component reusability.
  • the specification is distributed via the communications server 212 to one or more developers 404, 404', 404" (generally, 404), who may be members, for example, of a distributed community of programmers such as the community 212 shown in FIG. 2.
  • the developers 404 are unrelated to each other.
  • the developers may have no common employer, may be geographically dispersed throughout the world, and in some cases have not previously interacted with each other.
  • the developers 404 may have participated in one or more competitions, and/or have had previously submitted software artifacts subject to reviews. This approach allows an entity 208 to gain access to a large pool of qualified software developers.
  • the communication can occur over a communications network such as the network 112 (FIG. 1), such as via an email, instant message, text message, a posting on a web page accessible by the web browser 116, through a news group, facsimile, or any other suitable communication.
  • the communication of the specification can be accompanied by an indication of a prize, payment, or other recognition that is available to the designer(s) that submit selected software design(s).
  • the amount and/or type of payment may change over time, or as the number of participants increases or decreases, or both.
  • multiple designers may be rewarded with different amounts, for example a larger reward for the best design, and a smaller reward for second place.
  • the number of designers receiving an award can be based on, for example, the number of designers participating in the design project, or other similar attributes.
  • the recipients of the specification can be selected by various means.
  • members of the community may have expressed interest in participating in a development project, whereas in some cases the individuals are selected based on previous performances in coding competitions, prior development projects, or other methods of measuring the programming skill of a software developer.
  • the members of the distributed community of programmers may be programmers who have previously participated in an on-line programming competition.
  • the programming skills of the participants may have been rated according to their performance, either individually, as a team, or in relation to other programmers, and the ratings may be used to determine which programmers are eligible to receive notification of a new specification or respond to a notification.
  • the facilitator 400 moderates a collaborative forum among the various participants (the external entity 208, the developers 404, etc.) to determine, discuss, or collaborate on design features.
  • the collaborative forum can consist of developers, customers, prospective customers, or others interested in the development of certain software.
  • the collaboration forum is an online forum where participants can post ideas, questions, suggestions, or other information. In some embodiments, only a subset of the forum members can post suggestions to the forum.
  • one or more developers 404 each develop software designs (STEPS 412, 412' and 412") in accordance with the specification.
  • the development of the software design can be done using any suitable development system, for example, the software development software 224 provided via the communication server 216, a development environment provided by the developer 404, or some combination thereof.
  • a developer 404 is satisfied that her design meets the specified requirements, and follows the structured development methodology 220, she submits her design e.g., via the communications server 216, facsimile, email, mail, or other similar methods.
  • a design review process (STEP 414) is used. This design review can take place in any number of ways.
  • the facilitator 400 can delegate the review process to one or more members of the distributed community of programmers, or an appointee of the entity.
  • the design review process includes one or more developers 404 acting as a design review board to review design submissions from software designers.
  • the design review board preferably has a small number of (e.g., less than ten) members, for example, three members, but can be any number.
  • the review board is formed for only one or a small number of related projects, for example three projects.
  • Review boards in some embodiments, could be formed for an extended time, but changes in staffing also can help maintain quality.
  • one member of the design review board members is selected as the primary review board member by the facilitator 400 and/or the project manager, the members of the review board, and/or the external entity requesting the software program.
  • the facilitator 400 or a representative of the facilitator 400 acts as the primary review board member.
  • the primary review board member is responsible for coordination and management of the activities of the board.
  • submissions for software designs are judged by the design review board.
  • the primary review board member screens the design submissions before they are reviewed by the other members of the design review board, to allow the rest of the review board to judge only the best of the submissions.
  • the screening process includes scoring the submissions based on the degree to which they meet formal requirements outlined in the specification (e.g., format and elements submitted).
  • scores are documented using a scorecard, which can be a document, spreadsheet, online form, database, or other electronic document.
  • the design review board may also, in some cases, verify the anonymity of the developers 404 such that their identities cannot be discerned from their submissions.
  • a screening review can determine whether the required elements of the design are included (e.g., class, use-case, and sequence diagrams, component specification, required algorithms, class stubs, and functional tests). The screening review can also determine that these elements appear complete. With regard to the class diagram, for example, and in particular the class definition, the screening review can determine any or all of that: (1) the class definition provides a descriptive overview of the class usage, (2) sub-packages have been created to separate functionality, (3) class scope matches class usage, (4) there is proper and effective use of programming techniques such as inheritance and abstraction, (5) interfaces are used properly, (6) suitable constructors are defined for the component, and that (7) class modifiers such as final and static, are appropriately used.
  • the screening review can also determine, for example, with regard to variable definitions, that: (1) variable scope is correctly defined, (2) type assignments are defined appropriately for balance between efficiency and flexibility, and (3) that all variables are defined with an initial value. Further, with regard to method definitions, for example, the screening review can determine that: (1) scope is correctly defined, (2) exceptions are handled and used appropriately, (3) modifiers are properly used, (4) return types are used, (5) method arguments are properly defined, and (6) that the application programming interface (API) as stated in the requirements specification is available.
  • API application programming interface
  • the screening review can also, for example, verify that use-case diagrams exist for all public methods in the design, and that sequence diagrams exist for each use case.
  • the screening review can also, for example, with regard to test cases, verify that functional test cases are provided for each sequence diagram, and that they appear to be appropriate for those diagrams.
  • the designs can take a number of forms, depending on the program specified. Typically, the specifications will include the requirements for the design.
  • the design requirements include class diagrams, which can be developed in the Unified Modeling Language (UML), for example using the Poseideon Computer Aided Software Engineering (CASE) tool, available from Gentleware AG of Hamburg, Germany.
  • UML Unified Modeling Language
  • CASE Poseideon Computer Aided Software Engineering
  • the design requirements also include use-case diagrams and sequence diagrams.
  • the design requirements also include a written component design specification describing the design, a list of required algorithms, and class stubs for the classes in the design.
  • the design requirements also include functional tests that can be used to test the program.
  • the functional tests are tests compatible with the JUnit testing infrastructure. JUnit is open source software for testing Java software, which is available from www.sourceforge.net.
  • the primary review board member informs the design review board that one or more submissions have passed the initial screening process (STEP 416), and the design review board then evaluates the design submissions in greater detail.
  • the design review board reviews the submissions based on requirements documented in the specification.
  • the design review board scores the submissions (STEP 418).
  • the scores are documented using a scorecard, which can be any form, including a document, spreadsheet, online form, database, or other electronic document.
  • the scores and reviews from the primary review board member and the other members of the design review board are aggregated into a final review and score.
  • the aggregation can comprise compiling information contained in one or more documents. Such aggregation can be performed by the primary review board member, the other members of the design review board, or in one exemplary embodiment, the aggregation is performed using a computer-based system which resides on the server 104 (FIG. 1). In some embodiments, the facilitator 400 or the primary review board member resolves discrepancies or disagreements among the members of the design review board.
  • the design with the highest combined score is selected as the winning design that will be used for implementation (STEP 420).
  • a prize, payment and/or recognition is given to the designer.
  • a portion of the payment to the designer is withheld until the end of the development review.
  • the designer may receive 75% of the payment and the end of the design review, and 25% is paid after the code review.
  • the designers that submit the second and third best designs may also receive payment, which in some cases may be less than that of the winning designer. Payments may also be made for creative use of technology, submitting a unique test case, or other such submissions.
  • the software developers can contest the score assigned to their design, program, or other submissions.
  • the posted design is assigned a difficulty level, or some similar indication of how difficult the external entity, facilitator 400 or some evaluator of the design, believes it will be to produce a software program or component that meets the requirements of the selected design.
  • the difficulty level assigned to a design may, in some cases, also factor in the effort believed to be necessary to complete the task, and the time allotted to complete the task.
  • the recognition awarded to the designer e.g., money, skill rating, etc.
  • the selected design may be adjusted based in part on the difficulty level associated with the specification.
  • the design review board in addition to reviewing the submissions, can identify useful modifications to the design that should be included into the design prior to entering the development phase.
  • the primary review board member documents the additional requirements, and communicates this information to the designer 404 who submitted the design.
  • the primary review board member aggregates the comments from the review board.
  • the developer 404 can update the design and resubmit it for review by the design review board. This process can repeat until the primary review board member believes the design has met all the necessary requirements.
  • the primary review board member notifies the facilitator 400, product manager, or external entity that such a design has passed the design review process.
  • the design can then be posted and/or distributed (STEP 422) to the community of developers 407 to solicit submissions for software programs that conform to the design.
  • the facilitator 400 can make the design available on a web site and/or a mailing list for implementation, and request components according to the design.
  • the entity develops the software design and provides the design to the facilitator 400 as input directly into the development process.
  • the facilitator 400 receives the design (STEP 424) and optionally initiates a review process as described above to confirm that the design meets the standards of the structured development methodology 220.
  • an entity wishing to maintain control of the design phase of the software development process e.g., architecture, platform, coding standards, etc.
  • can utilize internal or other resources such as business and systems analysts to develop a design that complies with their standards, and then utilize a distributed community of developers 212 to develop the end product.
  • this alternative maintains the design aspects of the software development process in-house, and "outsources" the manufacturing aspects of the development process such that the development domain 204 can use repeatable, structured development methods and the community of developers 212 to develop the software programs.
  • the entity 208 may only require the services of the development domain 204 to develop a software design, and subsequently use other resources such as in house programmers or off shore developers to develop the code.
  • the flexibility provided by maintaining multiple entry and exit points into and out of the development process allows external entities to decide, on a case by case or phase by phase basis whether to utilize the development domain 204 from start to finish, (i.e., specification through testing and support) or only use the domain 204 for specific phases of the process (i.e., development of code, development of a specification, development of a software design, testing, support, etc.).
  • the selected and approved design is posted or provided to members of the distributed community of programmers 212.
  • the design may be sent to the entire community or only selected members of the community.
  • the selection process can be based on any or a combination of suitable criteria, for example, without limitation, past performances in programming competitions, the quality of previously submitted software programs, involvement in the development of the design, or by specific request of the facilitator 400, entity 208, the designer that submitted the winning design, other designers, or other members of the community 212.
  • the communication of the design can be accompanied by an indication of a prize, payment, or other recognition that is available to the developer that submits a selected software program, and/or runners up. In some cases, the amount and/or type of payment may change over time, or as the number of participants increases or decreases.
  • Each developer 404 develops software code (STEPS 426, 426', and 426") meeting the requirements of the selected design, and when completed, submits the code for example to the facilitator 400 or the server.
  • the developers 404 may use a variety of coding techniques, languages, and development environments to develop the software, so long as the code meets, for example, the functional and architectural aspects dictated by the design and the quality and syntactical standards outlined by the structured development methodology 220.
  • the developers 404 may use the software development software 224 provided via the communication server 216 to assist with the development tasks. Because the development software 224 and development methodology 220 are both maintained within the development domain 204, many of the coding and quality control requirements of the methodology 220 can be built into the software 224, further assisting the developers 404 to develop quality code in an efficient manner.
  • the code review process includes one or more developers 404 acting as a code review board to review submitted software programs from software developers.
  • the code review board preferably has a small number of members (e.g., less than ten), for example, three members, but can be any number.
  • the code review board is formed for only one or a small number of related projects, for example three projects, and then disbanded to allow the members to participate in additional design review boards, code review boards, or participate as designers and/or developers themselves.
  • Review boards in some embodiments, could be formed for an extended time, but changes in staffing also can help maintain quality.
  • one member of the code review board members is selected as the primary code reviewer by the facilitator 404 and/or the project manager, the members of the review board, and/or the external entity requesting the software program.
  • the facilitator 400 or a representative of the facilitator 400 acts as the primary code board member.
  • the primary code board member is responsible for coordination and management of the activities of the board.
  • submissions of software programs are judged by the code review board.
  • the primary review board member screens the code submissions before they are reviewed by the other members of the code review board, to allow the rest of the code board to judge only the best of the submissions, for example, those that meet minimal requirements.
  • the screening process includes scoring the submissions based on the degree to which they meet formal requirements outlined in the selected design (e.g., format and elements submitted).
  • scores are documented using a scorecard, which can be a document, spreadsheet, online form, database, or other electronic document.
  • the code reviewer scores the code based on the extent to which: (1) the submitted code addresses the functionality as detailed in component design documents; (2) the submitted code correctly uses all required technologies (e.g. language, required components, etc.) and packages; (3) the submitted code properly implements required algorithms; (4) the submitted code has correctly implemented (and not modified) the public application programming interface (API) as defined in the design, with no additional public classes, methods, or variables.
  • API public application programming interface
  • the screening review can determine any or all of that: (1) all public methods are clearly commented; (2) required tags such as "@author,” “@param,” “@return,” “@throws,” and “@version” are included; (3) the copyright tag is populated; (4) the source code follows standard coding conventions for the Java language such as those published by Sun Microsystems; (5) a 4 space indentation is used in lieu of a tab indentation; and (6) all class, method and variable definitions found in the class diagram are accurately represented in the source code.
  • the code review can also, for example, verify that unit test cases exist for all public methods in the design, and each unit test is properly identified by a testing program.
  • the reviewer can evaluate the code based on the extent to which classes are implemented as defined in design documents (including, for example, modifiers, types, and naming conventions), and whether defined classes are implemented.
  • the reviewer can determine the extent to which all variables and methods are implemented as defined in the design documents (including, for example, modifiers, types, and naming conventions).
  • the reviewer can determine the extent to which the implementation properly maps class relationships.
  • the reviewer can further evaluate code based on a code inspection.
  • the reviewer can determine the extent to which the object types defined in the code are the best choices for the intended usage - for example whether a Vector type should have been used instead of an Array type.
  • the reviewer can determine the extent to which there are any needless loops, or careless object instantiation or variable assignment.
  • the review can also inspect the test cases. With regard to test cases, for example, the reviewer can determine the extent to which (1) the unit test cases thoroughly test all methods and constructors; (2) the unit test cases properly make use of setup and teardown methods to configure the test environment; (3) files used in unit test cases exist in the designated directory; (4) unit test cases do not leave temporary files on the file system after testing is complete. [0077]
  • the reviewer can run tests on the code using test cases, for example test cases developed by the developer 404, other developers, the reviewers, the facilitator 400, the entity 208, as well as others. The reviewer can even further score the code by conducting accuracy, failure, and stress tests. Accuracy tests test the accuracy of the resulting output when provided valid input. Accuracy tests can also validate configuration data.
  • Failure tests test for correct failure behavior when the component is provided with invalid input, such as bad data and incorrect usage. Stress tests test the component capacity for high- volume operation, but testing such characteristics as performance as throughput. The tests that fail are included in the evaluation of the component, for example as a score reduction. The reviewer can then assign an overall score to the component based on this evaluation.
  • the primary review board member informs the code review board that one or more submissions have passed the initial screening step (STEP 430), and the code review board can then evaluate the program submissions in greater detail.
  • the code review board can review the submissions based on design requirements documented in the selected design.
  • the code review board can then score the submissions (STEP 432) based on the results of the evaluations.
  • the scores are documented using a scorecard, which can be any suitable means, such as a document, spreadsheet, online form, database, or other electronic document.
  • the scores and reviews from the primary code board member and the other members of the code review board are aggregated into a final review and score.
  • aggregation can comprise compiling information contained in one or more documents. Such aggregation can be performed by the facilitator 400, the primary code board member, the other members of the code review board or in one exemplary embodiment, the aggregation is performed using a computer-based system which resides on the server 104 (FIG. 1). In some embodiments, the facilitator 400 or the primary review board member resolves discrepancies or disagreements among the members of the code review board. [0080] In one embodiment, the software program with the highest combined score is selected as the winning program (STEP 434) that will be delivered to the external entity 208 as a finished product (STEP 436). In some embodiments, a prize, payment and/or recognition is given to the software developer that submitted the winning program.
  • the programmers that submit the second and third best programs may also receive payment, which in some cases may be less than that of the winning programmer. Payments may also be made for creative use of technology, submitting a unique test case, or other such submissions. In some embodiments, the software developers can contest the score assigned to their programs, test cases, or other submissions.
  • the code review board in addition to reviewing the submissions, can identify useful modifications to the program that should be included into a selected software program prior to distribution.
  • the primary code review board member documents the additional requirements, and communicates this information to the developer 404 who submitted the code.
  • the primary code review board member aggregates the comments from the review board.
  • the developer 404 can update the program and resubmit it for review by the code review board. This process can repeat until the primary review board member believes the program has met all the necessary requirements and meets the standards specified in the structured development methodology 220.
  • the software may be updated with enhancements, post- delivery bug fixes, additional functionality, or modified to operate in additional computing environments or platforms after it has been delivered to one or more entity 208.
  • the domain 204 provides for the tracking and updating (STEP 438) of previously distributed software products, as described in co-pending U.S. Patent Application Serial No. 10/408,402, entitled “Method and Systems for Software Development” by Hughes, filed on April 7, 2003, and incorporated by reference in its entirety herein.
  • an entity commissions the development of a software component, and upon completion of the component, version 1 of the component is distributed to the entity 208.
  • a second entity 208 requests the development of a similar component that performs the same functionality, however to meet the specific request of the second entity, some modifications are made to the component.
  • a modification is, for example, an improvement (e.g., efficiency increase, smaller memory requirements), deletion (e.g., of an unneeded step or feature), and an addition (e.g., of a complimentary feature or function) to the component.
  • Another example of a modification is the integration of the component into another component (e.g., a larger component).
  • a new version of the component version 1.1, for example is developed and distributed to the second entity 208.
  • a message is sent to the first entity 208 stating that an updated version of the component is available.
  • the costs for developing the newer version of the component can be shared among the recipients of the original component (version 1) who wish to receive the new version, as well as the entity that initiated the development of the new version. Additionally, in some embodiments the entity 208 that requested the development of the new version is compensated for licenses / sales of copies of the second version of the component.
  • the developers 404 submit one or more test cases in addition to submitting the completed software program. The purpose of the test cases is to provide sample data and expected outputs against which the program can run, and the actual output of which can be compared to the expected outputs.
  • a program that calculates amortization tables for loans may require input data such as an interest rate, a principal amount, a payment horizon, and a payment frequency.
  • Each data element may need to be checked such that null sets, zeros, negative numbers, decimals, special characters, etc. are all accounted for and the appropriate error checking and messages are invoked.
  • the mathematical calculations should be verified and extreme input values such as long payment periods, daily payments, very large or very small principal amounts, and fractional interest rates should also be verified.
  • one test case can be developed to check each of these cases, however in other versions, it may be beneficial to provide individual test cases for each type of error.
  • the multiple test cases can then be incorporated into a larger test program (e.g., a script, shell, or other high level program) and run concurrently or simultaneously.
  • a larger test program e.g., a script, shell, or other high level program
  • developers are encouraged to develop test cases as they are coding so that they can consider the bounding and error conditions as they code. It can be beneficial to use the test cases developed by one or more, or all, of the other submitters to test each of the submitted programs to cover as many error conditions as possible.
  • developers 404, 404' and 404" each submit software programs 502, 502' and 502" respectively to the development domain 204 in response to the communicated software design and/or specification referred to above.
  • the developers 404 In addition to submitting the programs, the developers 404 also submit one or more test cases 506, 506', and 506". For example, when DEVELOPER 1 404 submits PROGRAM 1 502, she also submits TEST CASE 1 A and TEST CASE IB, collectively 506. DEVELOPER 2 404' and DEVELOPER 3 404" do the same, such that after all three developers 404 have completed their submission, the development domain 204 includes a submission pool 508 comprising three submitted programs and six test cases.
  • test cases 506 * and 506" submitted by DEVELOPER 2 404' and DEVELOPER 3 404" respectively address cases or data not contemplated by DEVELOPER 1 404. Therefore, it can be advantageous to run each test case submitted by all of the developers against each of the submitted programs in an attempt to identify all potential faults of each submitted program. In some versions, a subset of the submitted test cases may be eliminated from the submission pool 508, or not used, for example, because they are duplicative, do not test necessary features, or are incorrect.
  • test cases in the submission pool 508 can be used to test the submitted programs. Because the programs are tested more rigorously (i.e., using a suite of test cases submitted by numerous developers) the quality of the resulting programs is likely to be greater than that of programs tested only by those that developed the selected program.
  • test cases in the submission pool 508 are applied to the submitted programs 502, 502', 502". In some cases, all of the test cases in the pool 508 are applied to every submitted program, whereas in some versions only a subset of the submitted test cases are used. In some embodiments, certain programs may be eliminated from contention by running a first test case against it, such that subsequent test cases are not necessary. In some versions, each application of test case to a program results in a score 604. The scores 604 for each application of test case to submitted program can then be tabulated and aggregated into a combined, or overall score for that particular program.
  • test cases have a higher or lower weight than others such that the scores for a particular test case may be more indicative of the overall quality of the program, or the results are more meaningful.
  • the scores may be binary - i.e., a passed test receives a score of "1" and a failed test receives a score of "0.”
  • the tabulation and aggregation can be automated on the server 104.
  • developers that submit designs and/or developed code are rated based on the scores of their submissions. Jhe ratings are calculated based on the ratings of each developer prior to the submission, the assigned difficulty level of the design or program being submitted, and the number of other developers making submissions.
  • a submission could be one design, program, or other computer software asset, or in some cases a number of different assets.
  • a skill rating is calculated for each developer based on each developer's rating prior to the submission and a constant standard rating (e.g., 1200), and a deviation is calculated for each developer based on their volatility and the standard rating.
  • the expected performance of each developer submitting a design or program is calculated by estimating the expected score of that developer's submission against the submissions of the other developers' submissions, and ranking the expected performances of each developer.
  • the submission can be scored by an reviewer using any number of methods, including, without limitation, those described above.
  • each developer is ranked, and an actual performance is calculated based on their rank for the current submission and the rankings of the other developers.
  • the submissions from other developers used for comparison are for the same program.
  • the submissions from other developers are submissions that are of similar difficulty or scope.
  • a competition factor also can be calculated from the number of developers, each developer's rating prior to the submission of the design or program, the average rating of the developers prior the submissions, and the volatility of each developer's rating prior to submission.
  • Each developer can then have their performance rated, using their old rating, the competition factor, and the difference between their actual score and an expected score. This performance rating can be weighted based on the number of previous submissions received from the developer, and can be used to calculate a developer's new rating and volatility. In some cases, the impact that a developer's performance on one submission may be capped such that any one submission does not have an overly significant effect on a developer's rating.
  • a developer's score may be capped at a maximum, so that there is a maximum possible rating.
  • the expected project performance of each developer is calculated by estimating the expected performance of that developer against other developers and ranking the expected performances of each participant.
  • the submissions and participants can be scored by the facilitator 400, the entity 208, a review board member, or automatically using the software residing, for example, on the server 104 using any number of methods.
  • the number of points awarded depends on the difficulty in completing the submission.
  • the server may also adjust the point value awarded to a developer based on the time that the server receives the developer's submission compared to the time that the server provides the design.
  • PT is the time a developer spends developing a submission (i.e., producing the computer code)
  • TT is the total time allocated for coding all submissions
  • MP is the maximum points available for developing the submission.
  • this number of points e.g. 63% of the total available
  • partial credit is given for partially correct submissions.
  • the submission receives the number of points based on a percentage of test data.
  • the number of points received is scaled based on the percentage of test data with which the submission executes correctly.
  • the server tracks the points awarded to each developer for each round. The server may also calculate a running total for each developer, summing the points awarded in each completed round of the competition. This running total can be used to eliminate low-scoring developers.
  • the server may assign a rating to the developer.
  • the assignment of the rating occurs when a competition or tournament is complete.
  • a competition may consist of one round, or may consist of multiple rounds, such as in a tournament.
  • the server assigns a rating to each developer after awarding total points to each developer.
  • the competition includes multiple rounds (e.g., a tournament)
  • the server assigns a rating to each developer after the developers complete the tournament.
  • the ratings provide a metric that can be used to chart a developer's skill, knowledge, and ability.
  • the server bases the developer's rating on the total number of points awarded to the developer during the competition (e.g., having one or multiple rounds).
  • the rating can be a score, a grade, or any other type of rating technique.
  • the external entity 208 is interested in receiving the developed design or the code, as well as obtaining developers' ratings, and in some cases only the ratings.
  • the external entity 208 may ask developers to participate in the development process just so that the developers are rated, and their skills can be objectively evaluated for future projects of greater value, or to determine which developers are more skilled.
  • the requestor could, in addition, have some interest in the developed design or code, and may have some interest in using the developed intellectual asset for its business or otherwise.
  • design reviewer(s) in the design review process and/or code reviewer(s) in the code review process There can be a significant benefit to using personnel who are rated highly, using the process described above, as design reviewer(s) in the design review process and/or code reviewer(s) in the code review process.
  • One of the traditional problems with conducting code reviews has been that the abilities of the reviewers were not established. Review by a poorly skilled developer can result in an inadequate review.
  • the process By using the process to select as reviewers only developers with sufficient skill (as determined by the process), the process itself insures its success.
  • this software development process is adopted by a software development group within an organization.
  • the development performed by the group is conducted using this process.
  • Each developer in the group has a rating, and the developers work to improve and/or maintain their ratings.
  • Developers who have high ratings can participate in reviews (e.g., the design review process or the code review process).
  • developers receive additional benefits and or compensation for achieving a high rating.
  • developers can receive additional benefits and/or compensation for such participation in a review process.
  • the requestors in this example are product or program managers, charged with directing the software development.
  • an outside organization such as a consultant can use the system and methods described above to evaluate and rate the development competencies of a development group.
  • the consultant can rate the developers not only against themselves, but against other developers affiliated with other organizations who have participated or are participating in the system.
  • the evaluator provides the service of evaluation and reporting as described above.
  • One benefit to this approach is that the scoring of the intellectual assets are more likely to be unbiased if the reviewers are not personally known to the developers, and comparing the skills of any one developer against a large pool of developers provides a more accurate representation of that developers skill level with respect to his or her peers.
  • the server 104 can include a number of modules and subsystems to facilitate the communication and development of software specifications, designs and programs.
  • the server 104 includes a communication server 704.
  • a communication server 704 is a web server that facilitates HTTP HTTPS and other similar network communications over the network 112, as described above.
  • the communication server 704 includes tools that facilitate communication among the distributed community of programmers 212, the external entity 208, the facilitator 400, and the members of the review board(s) (commonly referred to as "users").
  • Examples of the communication tools include, but are not limited to, a module enabling the real-time communication among the developers 404 (e.g., chat), news groups, on-line meetings, and document collaboration tools.
  • the facilitator 400 and/or the external entity 208 can also use the communication server 704 to post design or specifications for distribution to the distributed community of programmers 212.
  • the server 104 also includes a software development environment 702 to facilitate the software development domain 204 and the design and development process, for example, and the subsystems and modules that support the domain 204.
  • the server 104 can include a development posting subsystem 708, a management subsystem 712, a review board subsystem 714, a testing subsystem 716, a scoring subsystem 720, a methodology database 724, and a distribution subsystem 728.
  • the development posting subsystem 708 allows users of the system to post specifications, submit designs, post selected designs, submit software programs and test cases, and post selected software programs for distribution.
  • the posting subsystem 708 identifies the users based on their role or roles, and determines which functions can be accessed based on individual security and access rights, the development phase that a project is currently in, etc. For example, if a particular project is in the design development phase, the posting subsystem 708 can determine that the external entity sponsoring the project has read/write access to the specification, and can re-post an updated specification if necessary.
  • the facilitator 400 may have read access to the specification, as well as access to other specifications attributed to other external entities they may support. In some embodiments, the entire distributed community of programmers may be able to view all of the currently pending specifications, however the posting subsystem may limit full read access to only those developers meeting one or more skill or rating criteria, as described above.
  • the development posting subsystem 708 also enables the server 104 or other participants to communicate with potential developers to promote development projects and grow the community of programmers that participate in the development process.
  • the development posting subsystem 708 displays an advertisement to potential developers.
  • the advertisement describes the project using text, graphics, video, and/or sounds. Examples of communication techniques include, without limitation, posting these ads on the server's web site, displaying statistics about the project (e.g., planned royalties paid to developers, developers who are participating in this project, development hours available per week).
  • the development posting subsystem 708 accepts inquiries associated with development projects.
  • the development posting subsystem 708 suggests development opportunities to particular developers.
  • the development posting subsystem 708 may analyze, for example, the rating of each member of the distributed community, previous contributions to previous development projects, the quality of contributions to previous component development projects (e.g., based on a score given to each developer's submission(s) as discussed above), and current availability of the developer to participate.
  • the server 104 also includes a management subsystem 712.
  • the management subsystem 712 is a module that tracks the progress of design and development projects using the software development environment 204.
  • the management subsystem 712 also facilitates the enrollment of new users of the system, and assigns the appropriate security and access rights to the users depending on the roles they have on the various projects.
  • the management subsystem 712 can also compile and track operational statistics of the software development environment 204 and users of the system. For example, to determine the appropriate compensation to be awarded to a developer submitting a wining design, the management subsystem 712 may review previously completed projects and assign a similar cash award.
  • the management subsystem 712 can review information about individual programmers to determine those developers who have historically performed well on like projects. In addition, the management subsystem 712 may be used to analyze overall throughput times necessary to develop operational programs from a specification provided by an external entity. This can assist users of the system in setting the appropriate deliverable dates and costs associated with new projects.
  • the server 104 also includes a review board subsystem 714.
  • the review board subsystem 714 allows review board members, external entities, the facilitator, and in some cases developers in the distributed community to review submissions from other developers, as described above.
  • the communication server 704, the development posting subsystem 708, the management subsystem 712, the review board subsystem 714, the testing subsystem, the scoring subsystem, and the methodology database reside on the server 104.
  • these components of the software development environment 204 can reside on other servers or remote devices.
  • the server 104 additionally includes a testing subsystem 716.
  • the testing subsystem 716 enables the testing of the submitted programs, applications and/or components.
  • the testing server 708 is used by the review boards, the facilitator 400, and/or the external entity 208 to review, evaluate, screen and test submitted designs and software programs.
  • the testing subsystem 716 can also execute test cases developed and submitted by the developer 404 against some or all of the submitted programs, as described above.
  • the testing subsystem 716 may execute an automated test on the component or application, such as to verify and/or measure memory usage, thread usage, machine statistics such as I/O usage and processor load. Additionally, the testing subsystem 716 can score the component by performance, design, and or functionality.
  • the testing subsystem 716 can be a test harness for testing multiple programs simultaneously.
  • the server 104 also includes a scoring subsystem 720. In one embodiment, the scoring subsystem 720 calculates scores for the submissions based on the results from the testing subsystem 716, and in some embodiments ratings for each participant in one or more coding competitions, previous development submissions, or both. In other embodiments, the scoring subsystem 720 can calculate ratings for developers based on their contributions to the project. [00110]
  • the server 104 also includes a methodology database 724.
  • the methodology database 724 stores data relating to the structured development methodology 220. In one embodiment, the methodology 220 may stipulate specific inputs and outputs that are necessary to transition from one phase of the development project to the next.
  • the methodology 200 may dictate that, in order to complete the specification phase of the project and being the design phase, a checklist of items must be completed.
  • the methodology database 724 may store sample documents, designs, and code examples that can be used as templates for future projects, and thus impose a standardized, repeatable and predictable process framework on new projects. This standardization reduces the risks associated with embarking on new software development projects, shortens the overall duration of new development projects, and increases the quality and reliability of the end products.
  • the server 104 also includes distribution subsystem 728.
  • the distribution subsystem 728 can track and store data relating to software products (e.g., specifications, designs, developed programs) that have been produced using the domain 204.
  • the distribution subsystem 728 includes descriptive information about the entity 208 that requested the product, the entry and exit points of the domain 204, significant dates such as the request date, and the delivery date, the names and/or nicknames of the developers that participated in the development of the product.
  • the distribution subsystem 728 can also include detailed functional information about the product such as technology used to develop the product, supported computing environments, as well as others. In some embodiments, previously distributed software products may be updated or patched, as described above.
  • the distribution subsystem 728 facilitates the identification of the entity or entities 208 that may have older versions of the product, and subsequent communication and distribution of updated versions, where applicable.
  • the distribution subsystem 728 can also function as a source code management system, thereby allowing various versions of previously developed software products to branch into distinct software products having a common provenance.
  • the cooperatively developed product can be any sort of tangible or intangible object that embodies intellectual property.
  • the techniques could be used for computer hardware and electronics designs, or other designs such as architecture, construction, or landscape design.
  • Other non-limiting examples for which the techniques could be used include the development of all kinds of written documents and content such as documentation and articles for papers or periodicals (whether on-line or on paper), research papers, scripts, multimedia content, legal documents, and more.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Business, Economics & Management (AREA)
  • Entrepreneurship & Innovation (AREA)
  • Economics (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Operations Research (AREA)
  • Quality & Reliability (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Stored Programmes (AREA)

Abstract

L'invention concerne des procédés et un système pour le développement de logiciel. Selon une variante, on décrit un procédé de développement de logiciel qui se décompose comme suit : communiquer une spécification pour la conception d'un programme logiciel à une première pluralité de développeurs, en réponse à la spécification, recevoir une conception de la part d'un sous-ensemble de ces développeurs, faciliter une revue de conception pour l'évaluation des conceptions, sélectionner une conception selon ladite évaluation, communiquer la conception sélectionnée à une seconde pluralité de développeurs, en réponse à la spécification, recevoir des programmes logiciels de la part d'un sous-ensemble de ces développeurs, faciliter une revue de programme logiciel pour l'évaluation des programmes logiciels, et sélectionner un programme logiciel sur la base de ladite évaluation. Selon une autre variante, on décrit un procédé pour le développement distribué de logiciel, qui se décompose comme suit : fournir un système de développement logiciel à une communauté distribuée de développeurs, le système comprenant un logiciel de développement logiciel qui utilise une méthodologie de développement structurée, un serveur de communication, et un conseil de revue permettant de déterminer la qualité du logiciel soumis; accepter une demande de création de programme logiciel ; et faciliter le développement du programme par un sous-ensemble de la communauté distribuée de développeurs en utilisant le système considéré.
EP05705832A 2004-01-15 2005-01-14 Systemes et procedes pour le developpement de logiciel Ceased EP1716483A4 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP11001173A EP2333657A1 (fr) 2004-01-15 2005-01-14 Système et procédés de développement logiciel

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US53676004P 2004-01-15 2004-01-15
PCT/US2005/001483 WO2005069902A2 (fr) 2004-01-15 2005-01-14 Systemes et procedes pour le developpement de logiciel

Publications (2)

Publication Number Publication Date
EP1716483A2 true EP1716483A2 (fr) 2006-11-02
EP1716483A4 EP1716483A4 (fr) 2008-07-16

Family

ID=34807045

Family Applications (2)

Application Number Title Priority Date Filing Date
EP05705832A Ceased EP1716483A4 (fr) 2004-01-15 2005-01-14 Systemes et procedes pour le developpement de logiciel
EP11001173A Ceased EP2333657A1 (fr) 2004-01-15 2005-01-14 Système et procédés de développement logiciel

Family Applications After (1)

Application Number Title Priority Date Filing Date
EP11001173A Ceased EP2333657A1 (fr) 2004-01-15 2005-01-14 Système et procédés de développement logiciel

Country Status (5)

Country Link
EP (2) EP1716483A4 (fr)
CN (2) CN101091173A (fr)
CA (1) CA2553220A1 (fr)
RU (1) RU2006129494A (fr)
WO (1) WO2005069902A2 (fr)

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008046223A1 (fr) * 2006-10-18 2008-04-24 Iscopia Software Inc. Outil logiciel permettant d'écrire un logiciel pour la gestion de qualification en ligne
CN102456177A (zh) * 2010-10-27 2012-05-16 镇江华扬信息科技有限公司 软件项目管理系统
US8627280B2 (en) * 2011-06-20 2014-01-07 Microsoft Corporation Multi-tenant collaborative review service
EP2549378A1 (fr) * 2011-07-20 2013-01-23 2Csi Procédé de spécification d'un produit logiciel, et serveur mettant en oeuvre un tel procédé
US9606894B2 (en) * 2015-03-04 2017-03-28 International Business Machines Corporation Collaborative collection of diagnostics data of software programs
IN2015DE02745A (fr) 2015-09-01 2015-09-25 Hcl Technologies Ltd
CN105786488A (zh) * 2016-02-17 2016-07-20 姚晓宁 一种移动互联网定向流量sdk开发系统及其开发方法
CN105843602A (zh) * 2016-03-16 2016-08-10 中国银行股份有限公司 一种用于软件开发的方法和系统
US11720330B2 (en) 2016-10-17 2023-08-08 Engineer.ai Corp. Application development involving instant protoyping
CN108268293B (zh) * 2016-12-29 2021-11-02 广东中科遥感技术有限公司 移动app快速原型演示的方法
CN107273124A (zh) * 2017-06-12 2017-10-20 北京奇虎科技有限公司 软件运行性能信息的处理方法、装置及系统
US10467062B1 (en) * 2019-03-11 2019-11-05 Coupang, Corp. Systems and methods for managing application programming interface information
US20230418564A1 (en) 2020-06-16 2023-12-28 Engineer.ai Corp Systems and methods for creating software
CN112668869A (zh) * 2020-12-25 2021-04-16 平安科技（深圳）有限公司 一种监控软件开发过程的方法、装置、终端及存储介质
CN113434121A (zh) * 2021-06-29 2021-09-24 上海律信信息科技有限公司 一种基于加密算法的软件开发系统
WO2023129677A1 (fr) * 2021-12-30 2023-07-06 Harman International Industries, Incorporated Système et procédé pour une certification d'application
CN117033287B (zh) * 2023-10-08 2024-02-13 易方信息科技股份有限公司 基于sdk包的多总线通信方法、系统、设备以及存储介质

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6662194B1 (en) * 1999-07-31 2003-12-09 Raymond Anthony Joao Apparatus and method for providing recruitment information
US20010049615A1 (en) * 2000-03-27 2001-12-06 Wong Christopher L. Method and apparatus for dynamic business management
US6658642B1 (en) * 2000-06-21 2003-12-02 International Business Machines Corporation System, method and program product for software development
US6569012B2 (en) 2001-01-09 2003-05-27 Topcoder, Inc. Systems and methods for coding competitions

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
No further relevant documents disclosed *
See also references of WO2005069902A2 *

Also Published As

Publication number Publication date
EP2333657A1 (fr) 2011-06-15
CA2553220A1 (fr) 2005-08-04
WO2005069902A3 (fr) 2007-01-04
EP1716483A4 (fr) 2008-07-16
CN102880455A (zh) 2013-01-16
CN101091173A (zh) 2007-12-19
WO2005069902A2 (fr) 2005-08-04
RU2006129494A (ru) 2008-02-20

Similar Documents

Publication Publication Date Title
US7778866B2 (en) Systems and methods for software development
US8776042B2 (en) Systems and methods for software support
US20060248504A1 (en) Systems and methods for software development
EP2333657A1 (fr) Système et procédés de développement logiciel
US20070220479A1 (en) Systems and methods for software development
US10783458B2 (en) Systems and methods for screening submissions in production competitions
US7292990B2 (en) System and method for software development
US20070250378A1 (en) Systems and methods for conducting production competitions
US20100030626A1 (en) Distributed software fault identification and repair
US20080196000A1 (en) System and method for software development
US8073792B2 (en) System and method for content development
US8209238B2 (en) Systems and methods for graphic design development
US20100022306A1 (en) System and method for conducting competitions
US8909541B2 (en) System and method for manipulating success determinates in software development competitions
Saripalli et al. Finding common denominators for agile software development: a systematic literature review
Alder Instead of the Wrecking Ball

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20060814

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IS IT LI LT LU MC NL PL PT RO SE SI SK TR

AX Request for extension of the european patent

Extension state: AL BA HR LV MK YU

PUAK Availability of information related to the publication of the international search report

Free format text: ORIGINAL CODE: 0009015

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 15/02 20060101AFI20070426BHEP

A4 Supplementary search report drawn up and despatched

Effective date: 20080618

RIC1 Information provided on ipc code assigned before grant

Ipc: G06F 9/44 20060101AFI20080612BHEP

17Q First examination report despatched

Effective date: 20090206

APBK Appeal reference recorded

Free format text: ORIGINAL CODE: EPIDOSNREFNE

APBN Date of receipt of notice of appeal recorded

Free format text: ORIGINAL CODE: EPIDOSNNOA2E

APBR Date of receipt of statement of grounds of appeal recorded

Free format text: ORIGINAL CODE: EPIDOSNNOA3E

APAF Appeal reference modified

Free format text: ORIGINAL CODE: EPIDOSCREFNE

REG Reference to a national code

Ref country code: DE

Ref legal event code: R003

APBT Appeal procedure closed

Free format text: ORIGINAL CODE: EPIDOSNNOA9E

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION HAS BEEN REFUSED

18R Application refused

Effective date: 20140325