AU2004218661A1 - A system and method for linguistic assessment of written text - Google Patents

Description

EDITORIAL NOTE Page number 2 is the first page of Description.

A system and method for linguistic assessment of a Swritten text Description The invention addresses the need for objective feedback on the linguistic aspects of one's writing, and offers the opportunity to assess one's writing skills, identify areas for improvement, and measure progress over time.

CThe invention is a method and system that utilises a database to capture 0 submissions of written text from a plurality of authors and other submitters, storing unique identifiers with each submission.

The method and system quantitatively assesses aspects of a written text by a Oprocess of linguistic analysis, comparison, measurement, and mathematical Ncalculations. Aspects include cohesion, syntactic fluency, sophistication, variety, vigour, complexity, linguistic quality, and readability, as well as consistency of these measures within a long written work.

It produces statistical values that are stored for presentation, and presents the quantitative results of the analysis at a computer workstation either by numerical value, graphically or both. At the submitter's option, the results are either presented on their own or in comparison with benchmarks for the type of work submitted. The system enables the submitter to review both the original written submission and the quantitative results of the submission, and to enter unique comments to be stored with the submission.

Furthermore, the system enables comparison of different submissions or resubmissions by the same submitter so as to gauge progress over time. A system of date stamping is used to facilitate this process.

In one embodiment, the system enables the recording of sets of results for class groups over a period of time, including appended comments for individual works in the set. In so doing, it helps educators in particular to undertake a more objective assessment of written texts, by bringing consistent and comparable quantitative analysis to an essentially subjective assessment task. Results of different class sets can also be averaged and compared. For educational applications, the benchmark would be set in accordance with curriculum requirements of educational authorities.

It is an object of the invention to create a system that will operate on all present computer operating system platforms, either on a single computer or a network of computers, as well as an internet-based service.

Throughout this specification and the claims below, unless the context requires otherwise, a stated integer or group of integers shall not be at the exclusion of any other integer or group of integers.

Objects of the invention

O

1. It is an object of the invention to objectively assess linguistic quality, style and readability of written text.

O 2. It is another object of the invention to determine a set of measures that can be used to assess cohesion, syntactic fluency, consistency, Osophistication, variety, vigour, complexity, and other aspects of linguistic quality and readability.

I3. It is another object of the invention to provide comparison of the written Itext to appropriate benchmarks, including those accepted by accredited 0 authorities.

4. It is another object of the invention to enable the assessment of different categories of writing against respective category benchmarks.

It is another object of the invention to provide feedback to facilitate selfediting and assist writers to identify areas for improvement in their writing.

6. It is another object of the invention to enable the assessment of writing consistency throughout a written text.

7. It is another object of the invention to provide style guidelines for contributors of written content for web-sites, magazines, etc, where consistency of writing is an issue.

8. It is another object of the invention to create sets of submitters to enable the average of a set to be calculated and enable an individual submitter's results to be compared to the set's average results.

9. It is another object of the invention to develop a computer-based application program which can be offered in a variety of formats including stand-alone, as a plug-in for word processing applications, and via the internet.

It is another object of the invention to create a system that will operate on all present computer operating system platforms, either on a single computer or a network of computers.

Summary of Claims The following are the main claims of the invention. The full list of claims appears later in this document.

1. A method and computer system that provides a statistical assessment of written text submitted by a plurality of persons, and which measures and reports aspects of its linguistic quality, and/or style, and/or readability, said system comprising submission, analysis, database and reporting.

2. A method and computer system, that assesses syllables and counts o 5 the number of syllables per word in order to assess the number of O difficult words and easy words in a text, as a basis for determining Scomplexity and grade level, said system comprising any or all of Ssubmission, analysis, database and reporting.

10.A method and computer system that measures word usage relative to 0common words and phrases sourced from published corpus in order to 0determine the level of the submitted author's word-craft ability, said 0_ system comprising any or all of submission, analysis, database and N reporting.

"z 18. A method and computer system for determining writing style using algorithms which calculate cohesion, syntactic fluency, consistency, sophistication, variety, vigour, complexity, linguistic quality and readability.

Embodiment For a better understanding of the invention, it will be described in relation to specific embodiments but it is not intended that the invention should be necessarily limited to any specific statement made herein or any of the specific details at least in so far that the broader concept of the invention is conceived. The embodiment is described by way of example only and modifications are possible within the scope of the invention.

An example of use of the Linguistic Assessment System is in a school environment where teachers assess the word-craft and English writing skills of students.

The role of teachers includes instruction and assessment of students' literacy in a manner prescribed by curriculum developers within the education system.

Strict guidelines are given to teachers include benchmarks indicative of the writing skills students are expected to achieve at each year level.

Competence is assessed using measures covering grammar, linguistic quality, word usage, clarity, and readability.

Teachers set course work for students to complete and submit for assessment. Students are then informed of areas in need of attention.

Currently, teachers of English assess and mark submitted work in a subjective way. In contrast, the Linguistic Assessment System helps the teacher to assess student work in a more objective way. The system provides statistical data about a student's work using objective and quantifiable performance indicators and benchmark comparisons. This supports, rather than replaces, the teacher's subjective assessment of the grade level and quality of student work.

The system can also be used to measure and record a student's comparative 5 performance relative to benchmarks and peer averages at each grade level.

O It also facilitates review of a student's progress over time.

OThe following steps show how the system would work in this environment.

1. The Linguistic Assessment System is installed in a computer accessible to the teacher.

2. Students produce and submit written work for assessment.

3. The submitter's reference details and written work are captured by the Linguistic Assessment System and analysed. Long works are also assessed for writing consistency.

4. The results of the analysis are presented to the submitter and/or teacher and stored for later review by the teacher.

The teacher accesses the information related to the particular student as required, and is able to append comments and a score value to the submission.

6. The teacher can view the analyses of work submitted by individual students over time in a way that identifies their progress.

7. The teacher can print reports to be used in discussion with the student or with other interested parties (teachers, parents, etc, who might be interested in the student's performance/progress).

8. The system can also be used to report the average results of a class set and to gauge an individual student's standing against the class average.

Another use of the Linguistic Assessment System is to screen candidates in writing games and competitions. Written responses submitted by authors are entered into a computer that applies the Linguistic Assessment System in order to assist in the assessment of submitted works.

The system assesses each work's linguistic quality, calculates a value, and ranks the works according to that value. Works of a higher linguistic quality would be ranked ahead of works of a lower linguistic quality.

Description of the Diagrams Diagrams depicting the various components of the invention are attached.

Diagram 1 System Overview

O

(A central database is fed by a submission sub-system which accepts written text, gives it a unique identifier, and stores both in the central database. It also O 5 passes the submitted text to the analysis sub-system for analysis. The results O of the analysis are stored in the central database against its unique identifier and details are presented to the submitter in line with the options selected.

OOne embodiment would include default and custom options for immediate presentation.

1 The system also comprises a query sub-system that enables submitters to 1 access their stored text and results of the analysis on demand, utilising either 0 default or custom options.

15 Diagram 2 Submission logging sub-system The diagram shows the path taken to store a submission into the database with unique identifiers, pass it for analysis by the analysis sub-system, and store the quantified results of the analysis in the database.

The submission sub-system also identifies long texts to be assessed for consistency, based on all the assessment measures described above. For long texts, the submission sub-system determines the total number of words in the text and divides the text into a number of blocks. The length of the text determines the number of blocks. Each block is processed independently by the analysis sub-system and collated in the database in block sequence with its own unique group identifier.

Diagram 3 Analysis sub-system This sub-system is made up of 4 modules, which assess different aspects of the written text. Each module passes the data comprising the results of the assessment into the database for storage against the unique identifier.

The syllable calculation module assesses the total number of words and the number of vital words in the written text. It then calculates the number of syllables in each word to determine the number of easy and hard words. It also counts the number of words by number of syllables in the word, namely the number of one syllable words, the number of two syllable words, the number of three syllable words, etc The sentence analysis module counts the total number of sentences in the text, and the number of sentences by the number of words in the sentence, namely the number of one word sentences, the number of two word sentences, the number of three word sentences, etc. Other aspects of sentence analysis include sentence structure, punctuation, and paragraphing.

The word usage analysis module is used to analyse common word usage, lazy words, style measures, pronouns, proper nouns, cohesive ties, conjunctions, adverbs, phrases and punctuation. The word usage analysis module utilises word lists in the database to count pre-defined words and O phrases in the text and to check spelling.

The complexity calculation module extracts necessary data deposited into the O 5 database by the 3 modules above and calculates values for readability, O complexity, and grade level, using specific algorithms, and stores the results of the calculation in the database.

Diagram 4- Inquiry sub-system 0This sub-system allows an authorised user to inquire on the results of \0previously submitted text. The user is given the option to select from a list of 00 elements of the analysis to construct their own report for review. The user is also given the option to choose from a number of presentation forma s, S 15 including graphic, spreadsheet and database, and to print the report a Srequired. User specified report formats can be saved for future use by the user.

The user is able to append comments to the file being reviewed. Comments are date stamped on entry and cannot be changed.

Diagram 5 Network version The system can be installed onto a local area network comprising a server and local and/or remote client computers. The database resides on the server and all client access to the stored data is via the server. Only authorised users are granted access to the server's database.

Diagram 6a Internet (WWW) version The Linguistic Assessment System and its database are installed on a secure remote computer linked to an Internet Service Provider's server which acts as a host for the web-based service.

Remote internet network access is routed through the ISP host to the Linguistic Assessment System computer via a secure login process.

Response to the inquiry is sent to the inquirer by the remote linguistic assessment system via the ISP.

Diagram 6b Internet (WWW) version The Linguistic Assessment System and related web-site reside on an Internet Service Provider's server. Authorised users access the application by logging onto the web-site via the internet.

Claims (21)

1. A method and computer system that provides a statistical assessment 0 of written text submitted by a plurality of persons, and which measures and reports aspects of its linguistic quality, and/or style, and/or readability, said system comprising Submission, Analysis, Database, and Reporting components.

2. A method and computer system, that assesses syllables and counts 0the number of syllables per word in order to assess the number of difficult words and easy words in a text, as a basis for determining Ncomplexity and grade level, said system comprising any or all of the following components: Submission N Analysis Database Reporting.

3. The methods and computer system claimed in claim 1 which enables the composing, capture, preparation, identification and storage in a central database of written text submitted by a plurality of submitters.

4. The method and computer system claimed in claims 1 and 2 which utilise statistical analyses, algorithms, and word lists in order to assess readability, complexity, grade level, and style measures of the submitted written text.

5. The computer system claimed in claims 1 and 2 which assesses submitted works individually in a comparable way, using an identical method of calculation and analysis.

6. The method and computer system claimed in either or both claims 1 and 2 which compares the quantitative results of the statistical assessment of submitted text against selected benchmark criteria.

7. The method and computer system claimed in claim 6 which enables the submitter to choose different benchmarks according to genre or set grade level work, as required for comparative analysis of the results.

8. For long written submissions, the method and computer system as claimed in claims 1 and 2 which assesses the consistency of the writing by dividing the submitted text into a number of segments, the number of segments depending on the length of the submitted text.

9. Each segment referred to in claim 8 is independently assessed, utilising the method and computer system claimed in claims 1, 2 and 3, then the measures from each segment are statistically analysed to determine the degree of consistency of the whole, and the results are presented to the submitter.

A method and computer system that measures word usage relative to o common words and phrases sourced from published corpus in order to O determine the level of the submitted author's word-craft ability and spelling ability, said system comprising any or all of the following: Submission Analysis Database Reporting. 00

11. The method and computer system claimed in claim 1 which presents (on demand and in a choice of formats the results of the analysis, including those results referred to in claims 2, 6, 7, 8, and 10 on a computer monitor's screen and in print.

12. The method and computer system claimed in claim 1 which enables comments to be stored with the work for later reference, after the work has been assessed as in claims 2, 5, 6, 7, 8, and 10, and after the opportunity to review the results as in claim 11.

13. The method and computer system claimed in claim 1 which tracks chronologically all of an individual writer's submissions, together with results of the analysis and any stored comments, as per claims 2 to 6 and 9 tol2, then compares these with appropriate benchmarks and with each other, as claimed in claims 7 and 8, in order to show change over time.

14. A method and computer system for storing and presenting results referred to in claims 4 and 9, and comparison data referred to in claims 6, 7 and 13, said results being stored and presented on demand as numerical values, graphically or both.

15. A method and computer system that enables the inquirer to select individual member submissions to make up a set, to have individual member results within a set averaged, and to record averaged results achieved by sets, said system comprising any or all of the following: Submission Analysis Database Reporting.

16. The method and computer system claimed in claim 15 which enables the inquirer to append comments to be stored with individual sets.

17. The method and computer system claimed in claim 15 which enables the averaged results of sets to be compared with selected benchmarks, as well as one to another.

18. A method and computer system for determining writing style using O algorithms which calculate any or all of cohesion, syntactic fluency, reading ease, consistency, sophistication, variety, strength, vigour, complexity, wordcraft, linguistic quality and readability. o O

19. The methods and computer systems claimed in claims 1 to 18 can function on a single computer or on a plurality of computers in a network, and operate on all present computer operating system platforms, and as an internet-based service.

020. A method and computer system for linguistic assessment of written \0text as described in claims 1 to 19 with reference to the attached oO diagrams. 15

21. A method and computer system described in claims 1, 2, 10 and 18, taken indvidually or in any combination.