AU2011218654A1 - Shape parameterisation for editable document generation - Google Patents

Abstract

SHAPE PARAMETERISATION FOR EDITABLE DOCUMENT GENERATION 5 Disclosed is a method of creating a modifiable shape in an electronic document from a freeform input shape. The method identifies (620) the freeform shape as one of a plurality of predefined template shapes and creates a modifiable shape (1450) having at least one shape modification point (1452, 1453). This shape is defined by the identified template shape, and the shape modification point is positioned to configure the modifiable shape 10 according to the freeform shape. The method then creates a control point for the modifiable shape the control point being (1451) manipulable when displayed on a display device to vary an appearance of the modifiable shape between the freeform shape and the identified template shape, and stores the modifiable shape and the control point in the electronic document. 5552872 2 P001898 specilodge r0IT cn (No ADIAI1IQOSI fiic~ Indrn

Description

S&F Ref: P001898 AUSTRALIA PATENTS ACT 1990 COMPLETE SPECIFICATION FOR A STANDARD PATENT Name and Address Canon Kabushiki Kaisha, of 30-2, Shimomaruko 3 of Applicant: chome, Ohta-ku, Tokyo, 146, Japan Actual Inventor(s): Michael John Blennerhassett James Austin Besley Address for Service: Spruson & Ferguson St Martins Tower Level 35 31 Market Street Sydney NSW 2000 (CCN 3710000177) Invention Title: Shape parameterisation for editable document generation The following statement is a full description of this invention, including the best method of performing it known to me/us: 5845c(5554952_1) - 1 SHAPE PARAMETERISATION FOR EDITABLE DOCUMENT GENERATION TECHNICAL FIELD The current invention relates to image processing and, in particular, to processing an input shape to find shape parameters and control points for the creation of an editable document. 5 BACKGROUND The proliferation of imaging technology combined with ever increasing computational processing power has led to many advances in the area of document analysis. A significant proportion of office documents are generated using structured text/graphics editing applications such as MicrosoftTM WordTM and MicrosoftTM 10 Powerpoint

TM

, both manufactured by Microsoft Corp. of U.S.A. In addition to formatted text editing, these text/graphics editing applications include basic figure drawing tools and options. Figure drawing options in a typical structured text/graphics editing application include freeform line drawing, template shapes and connectors (i.e., dynamic line objects 15 that connect to and/or between template shapes within a document). The text/graphics editing applications may also include colouring, filling, layering and grouping options for sets of objects. Many commonly used geometric shapes can be created using template shapes. A user may prefer to use a template shape rather than drawing the shape using freeform lines as this option can be faster, more accurate in terms of representation of the 20 desired shape, and easier to edit at a later time. The defining difference between a freeform shape and a template shape are control points, which allow faster modification of the template shape. The well-known MicrosoftTM AutoShapes set includes a number of examples of template shapes which can be manipulated within editing environments such as MicrosoftTM WordTM and PowerPointTM. Other template shapes may be found in 25 OpenOffice.orgTM editing applications such as the WriterTM and ImpressTM applications. 5552872_2 P001898_specilodge -2 An important class of applications used for document analysis, referred to as "scan to-editable" applications, process a bitmap representation of a document to generate an electronic version of the document that can be viewed and edited using such editing applications. The bitmap representation may have been created as a rendered version from 5 a text/graphics editing application, in which case the analysis may attempt to reconstruct the original content as accurately as possible. On the other hand the bitmap representation may have been generated by hand, in which case the analysis may attempt to create a representation that shows the intent of the author. An example would be where a user has drawn an approximately circular shape that the application converts to a circle. In some 10 cases, stroke data may be available in addition to the bitmap data. A second important class of applications, referred to as online applications, may process stroke data without bitmap data, for example in the case of hand-drawn input to a tablet PC, touch screen device, or gesture based input. These applications may generate an editable document with content that represents the input stroke data. 15 Methods exist that process a scanned page to detect a configuration for a template shape, determine parameters of the template, and generate a template shape configured by the determined parameters for output. However, template shapes vary between applications, so a particular system will generally handle only a limited set of template shapes. If a particular object cannot be matched to one of the template shapes accurately, 20 then the object is not matched. A non-matched object could persist as a bitmap object but may be vectorized. A need therefore exists for a method of processing objects that cannot be accurately matched to a template shape by a system such that they can be output as configurable shapes. 5552872_2 P001898_speci_lodge -3 SUMMARY According to one aspect of the present disclosure, there is provided a method of creating a modifiable shape in an electronic document from a freeform input shape, the method comprising the steps of: 5 identifying the freeform shape as one of a plurality of predefined template shapes; creating a modifiable shape having at least one shape modification point defined by the identified template shape, the at least one shape modification point being positioned to configure the modifiable shape according to the freeform shape; creating a control point for the modifiable shape the control point being 10 manipulable when displayed on a display device to vary an appearance of the modifiable shape between the freeform shape and the identified template shape; and storing the modifiable shape and the control point in the electronic document. Other aspects are also disclosed. BRIEF DESCRIPTION OF THE DRAWINGS 15 At least one embodiment of the present invention will now be described with reference to the following drawings, in which: Fig. I illustrates a system in which the arrangements to be described may be implemented; Fig. 2A shows a selection of MicrosoftTM Autoshapes; 20 Fig. 2B shows another selection of MicrosoftTM Autoshapes; Fig. 3 is a set of Autoshapes each with one template parameter showing how they individually deform; Fig. 4 is a flowchart of a first approach for generating an editable document using shape parameterisation; 25 Fig. 5 is a flowchart showing how a set of input freeforms are processed; Fig. 6 is a flowchart showing how each individual input freeform is processed; 5552872_2 P001898.specilodge -4 Fig. 7 is a flowchart showing how a parameterized shape with an appearance control point may be created; Fig. 8 is a flowchart showing how the points of a template shape way be parameterized using an appearance control point; 5 Fig. 9A illustrates segmentation of a freeform arrow shape; Fig. 9B illustrates editing of the shape of Fig. 9A by manipulation of a control point; Figs. 1OA and 10B show an example input freeform object, the closest template match and how the control points of the closest template match; 10 Fig. 11 illustrates a convex hull for an example input freeform; Fig. 12 illustrates a freeform segment group and a template segment; Figs. 13A and 13B show an example input freeform object, the closest template match and how the control points of the closest template match; Figs. 14A to 14E shows examples of a parameterized shape with control points and 15 an appearance control point in varied positions; Figs. 15A and 15B shows examples of a parameterized shape with an appearance control point and control points in varied positions; Fig. 16 shows examples of a parameterized shape with control points and an appearance control point; 20 Figs. 17A to 17C shows an example parameterized shape with control points and an appearance control point; Figs. 18A to 18C show an example group of parameterized shapes and connectors; Figs. 19A and 19B collectively form a schematic block diagram representation of an general purpose computer system upon which described arrangements can be practised; 25 Fig. 20 is a flowchart showing the second approach for shape parameterisation; Fig. 21 is a flowchart showing the third approach for shape parameterisation; and 5552872_2 P001898_speci_lodge -5 Appendix A is a table listing the names of each of the MicrosoftTM AutoShapes of Figs. 2A and 2B together with their corresponding drawing reference. DETAILED DESCRIPTION INCLUDING BEST MODE Fig. I illustrates a system 100 representing the environment in which the 5 arrangements to be described may be practiced. A document 110 is scanned by a scanner 120 to form an input scanned document image 130 that may be stored in memory as a bitmap image. The input scanned document image 130 is processed in a processing module 140 in accordance with one or more arrangements described herein. The processing module 140 may perform a number of bitmap image analysis processing stages 10 including improving the quality of the output geometric objects. A file description, or object representation 150 of the bitmap image 130 (i.e., the scanned document) is generated by the processing module 140 that includes figure content in the form of template shapes, connectors, and freeform elements with various styles such as line styles, fill styles and arrowheads. The file 150 may include figure elements 161 and 162 15 (graphical objects) that are suitable for editing using a structured text/graphics editing application executing on a suitable device, such as a computer 160, to which the file 150 may be input. The document 110 may be a compound document with a variety of content types. The content may include, but is not limited to, template shape elements 114, and 20 charts 115, in addition to non-figure elements such as text 111, tables 112 and (bitmap) images 113. The document 110 can be produced using a variety of equipment including printers, fax machines, projectors, or more traditional media such as pen on paper or whiteboard and may be an imperfect representation of an original electronic document. The scanner 120 may be a stand-alone scanner, a handheld scanner, or a scanner embedded 25 in a larger system, such as a multi-function printer. The scanner 120 may also be some other imaging device, such as a still or video camera, a mobile phone, tablet computing 5552872_2 P001898_specijodge -6 device or a personal organiser. The scanner 120 may also introduce noise into the input bitmap image 130. Examples of the processing module 140 include a computer, a multi functional printer, a mobile phone, or a personal organiser. When the scanner 120 is replaced by a mobile phone or tablet computer the input may be received using a touch 5 interface. In this situation a user of the device may draw an input shape on a screen using a finger and the device can capture the interaction. The image that is drawn can then be processed as a captured image. Figs. 19A and 19B depict a general-purpose computer system 1900, upon which the various arrangements described can be practiced. 10 As seen in Fig. 19A, the computer system 1900 includes: a computer module 1901; input devices such as a keyboard 1902, a mouse pointer device 1903, the scanner 120, a camera 1927, and a microphone 1980; and output devices including a printer 1915, a display device 1914 and loudspeakers 1917. Typically, the functions of the processing module 140 and computer 160 are performed by the computer module 1901 in association 15 with the various input and output devices. An external Modulator-Demodulator (Modem) transceiver device 1916 may be used by the computer module 1901 for communicating to and from a communications network 1920 via a connection 1921, for example as a source of the document image 130. The communications network 1920 may be a wide-area network (WAN), such as the Internet, a cellular telecommunications network, or a private 20 WAN. Where the connection 1921 is a telephone line, the modem 1916 may be a traditional "dial-up" modem. Alternatively, where the connection 1921 is a high capacity (e.g., cable) connection, the modem 1916 may be a broadband modem. A wireless modem may also be used for wireless connection to the communications network 1920. The computer module 1901 typically includes at least one processor unit 1905, and 25 a memory unit 1906. For example, the memory unit 1906 may have semiconductor random access memory (RAM) and semiconductor read only memory (ROM). The 5552872_2 P001898_speci_lodge -7 computer module 1901 also includes a number of input/output (1/0) interfaces including: an audio-video interface 1907 that couples to the video display 1914, loudspeakers 1917 and microphone 1980; an 1/0 interface 1913 that couples to the keyboard 1902, mouse 1903, scanner 120, camera 1927 and optionally a joystick or other human interface device 5 (not illustrated); and an interface 1908 for the external modem 1916 and printer 1915. In some implementations, the modem 1916 may be incorporated within the computer module 1901, for example within the interface 1908. The computer module 1901 also has a local network interface 1911, which permits coupling of the computer system 1900 via a connection 1923 to a local-area communications network 1922, known as a Local Area 10 Network (LAN). As illustrated in Fig. 19A, the local communications network 1922 may also couple to the communications network 1920 via a connection 1924, which would typically include a so-called "firewall" device or device of similar functionality. The local network interface 1911 may comprise an Ethernet T circuit card, a Bluetoothrm wireless arrangement or an IEEE 802.11 wireless arrangement; however, numerous other types of 15 interfaces may be practiced for the interface 1911. The I/O interfaces 1908 and 1913 may afford either or both of serial and parallel connectivity, the former typically being implemented according to the Universal Serial Bus (USB) standards and having corresponding USB connectors (not illustrated). Storage devices 1909 are provided and typically include a hard disk drive (HDD) 1910, upon 20 which for example the document image 130 and the output file 150 may be stored. Other storage devices such as a floppy disk drive and a magnetic tape drive (not illustrated) may also be used. An optical disk drive 1912 is typically provided to act as a non-volatile source of data. Portable memory devices, such as optical disks (e.g., CD-ROM, DVD, Blu-ray Disc TM), USB-RAM, portable, external hard drives, and floppy disks, for example, 25 may be used as appropriate sources of data to the system 1900. 5552872_2 P001898_speci lodge -8 The components 1905 to 1913 of the computer module 1901 typically communicate via an interconnected bus 1904 and in a manner that results in a conventional mode of operation of the computer system 1900 known to those in the relevant art. For example, the processor 1905 is coupled to the system bus 1904 using a connection 1918. 5 Likewise, the memory 1906 and optical disk drive 1912 are coupled to the system bus 1904 by connections 1919. Examples of computers on which the described arrangements can be practised include IBM-PC's and compatibles, Sun Sparcstations, Apple Mac" or a like computer systems. The methods of generating an editable document using shape parameterisation may 10 be implemented using the computer system 1900 wherein the processes of Figs. 4 to 8, and 20 and 21, to be described, may be implemented as one or more software application programs 1933 executable within the computer system 1900. In particular, the steps of the methods of editable document generation are effected by instructions 1931 (see Fig. 19B) in the software 1933 that are carried out within the computer system 1900. The software 15 instructions 1931 may be formed as one or more code modules, each for performing one or more particular tasks. The software may also be divided into two separate parts, in which a first part and the corresponding code modules performs the editable document generation methods and a second part and the corresponding code modules manage a user interface between the first part and the user. 20 The software may be stored in a computer readable medium, including the storage devices described below, for example. The software is loaded into the computer system 1900 from the computer readable medium, and then executed by the computer system 1900. A computer readable medium having such software or computer program recorded on the computer readable medium is a computer program product. The use of the 25 computer program product in the computer system 1900 preferably effects an 5552872_2 P001 898_speciJodge -9 advantageous apparatus for processing an input shape to find shape parameters and control points for the generation of an editable document. The software 1933 is typically stored in the HDD 1910 or the memory 1906. The software is loaded into the computer system 1900 from a computer readable medium, and 5 executed by the computer system 1900. Thus, for example, the software 1933 may be stored on an optically readable disk storage medium (e.g., CD-ROM) 1925 that is read by the optical disk drive 1912. A computer readable medium having such software or computer program recorded on it is a computer program product. The use of the computer program product in the computer system 1900 preferably effects an apparatus for editable 10 document generation. In some instances, the application programs 1933 may be supplied to the user encoded on one or more CD-ROMs 1925 and read via the corresponding drive 1912, or alternatively may be read by the user from the networks 1920 or 1922. Still further, the software can also be loaded into the computer system 1900 from other computer readable 15 media. Computer readable storage media refers to any non-transitory tangible storage medium that provides recorded instructions and/or data to the computer system 1900 for execution and/or processing. Examples of such storage media include floppy disks, magnetic tape, CD-ROM, DVD, Blu-ray Disc, a hard disk drive, a ROM or integrated circuit, USB memory, a magneto-optical disk, or a computer readable card such as a 20 PCMCIA card and the like, whether or not such devices are internal or external of the computer module 1901. Examples of transitory or non-tangible computer readable transmission media that may also participate in the provision of software, application programs, instructions and/or data to the computer module 1901 include radio or infra-red transmission channels as well as a network connection to another computer or networked 25 device, and the Internet or Intranets including e-mail transmissions and information recorded on Websites and the like. 5552872_2 P001898_speci-lodge - 10 The second part of the application programs 1933 and the corresponding code modules mentioned above may be executed to implement one or more graphical user interfaces (GUls) to be rendered or otherwise represented upon the display 1914. Through manipulation of typically the keyboard 1902 and the mouse 1903, a user of the computer 5 system 1900 and the application may manipulate the interface in a functionally adaptable manner to provide controlling commands and/or input to the applications associated with the GUI(s). Other forms of functionally adaptable user interfaces may also be implemented, such as an audio interface utilizing speech prompts output via the loudspeakers 1917 and user voice commands input via the microphone 1980. 10 Fig. 19B is a detailed schematic block diagram of the processor 1905 and a "memory" 1934. The memory 1934 represents a logical aggregation of all the memory modules (including the HDD 1909 and semiconductor memory 1906) that can be accessed by the computer module 1901 in Fig. 19A. When the computer module 1901 is initially powered up, a power-on self-test 15 (POST) program 1950 executes. The POST program 1950 is typically stored in a ROM 1949 of the semiconductor memory 1906 of Fig. 19A. A hardware device such as the ROM 1949 storing software is sometimes referred to as firmware. The POST program 1950 examines hardware within the computer module 1901 to ensure proper functioning and typically checks the processor 1905, the memory 1934 (1909, 1906), and a basic input 20 output systems software (BIOS) module 1951, also typically stored in the ROM 1949, for correct operation. Once the POST program 1950 has run successfully, the BIOS 1951 activates the hard disk drive 1910 of Fig. 19A. Activation of the hard disk drive 1910 causes a bootstrap loader program 1952 that is resident on the hard disk drive 1910 to execute via the processor 1905. This loads an operating system 1953 into the RAM 25 memory 1906, upon which the operating system 1953 commences operation. The operating system 1953 is a system level application, executable by the processor 1905, to 5552872_2 P001898_speci lodge - 11 fulfil various high level functions, including processor management, memory management, device management, storage management, software application interface, and generic user interface. The operating system 1953 manages the memory 1934 (1909, 1906) to ensure that 5 each process or application running on the computer module 1901 has sufficient memory in which to execute without colliding with memory allocated to another process. Furthermore, the different types of memory available in the system 1900 of Fig. 19A must be used properly so that each process can run effectively. Accordingly, the aggregated memory 1934 is not intended to illustrate how particular segments of memory are allocated 10 (unless otherwise stated), but rather to provide a general view of the memory accessible by the computer system 1900 and how such is used. As shown in Fig. 19B, the processor 1905 includes a number of functional modules including a control unit 1939, an arithmetic logic unit (ALU) 1940, and a local or internal memory 1948, sometimes called a cache memory. The cache memory 1948 typically 15 include a number of storage registers 1944 - 1946 in a register section. One or more internal busses 1941 functionally interconnect these functional modules. The processor 1905 typically also has one or more interfaces 1942 for communicating with external devices via the system bus 1904, using a connection 1918. The memory 1934 is coupled to the bus 1904 using a connection 1919. 20 The application program 1933 includes a sequence of instructions 1931 that may include conditional branch and loop instructions. The program 1933 may also include data 1932 which is used in execution of the program 1933. The instructions 1931 and the data 1932 are stored in memory locations 1928, 1929, 1930 and 1935, 1936, 1937, respectively. Depending upon the relative size of the instructions 1931 and the memory locations 1928 25 1930, a particular instruction may be stored in a single memory location as depicted by the instruction shown in the memory location 1930. Alternately, an instruction may be 5552872_2 P001898_speci_lodge - 12 segmented into a number of parts each of which is stored in a separate memory location, as depicted by the instruction segments shown in the memory locations 1928 and 1929. In general, the processor 1905 is given a set of instructions which are executed therein. The processor 1905 waits for a subsequent input, to which the processor 1905 5 reacts by executing another set of instructions. Each input may be provided from one or more of a number of sources, including data generated by one or more of the input devices 1902, 1903, data received from an external source across one of the networks 1920, 1902, data retrieved from one of the storage devices 1906, 1909 or data retrieved from a storage medium 1925 inserted into the corresponding reader 1912, all depicted in Fig. 19A. The 10 execution of a set of the instructions may in some cases result in output of data. Execution may also involve storing data or variables to the memory 1934. The disclosed arrangements use input variables 1954, which are stored in the memory 1934 in corresponding memory locations 1955, 1956, 1957. The arrangements produce output variables 1961, which are stored in the memory 1934 in corresponding 15 memory locations 1962, 1963, 1964. Intermediate variables 1958 may be stored in memory locations 1959, 1960, 1966 and 1967. Referring to the processor 1905 of Fig. 19B, the registers 1944, 1945, 1946, the arithmetic logic unit (ALU) 1940, and the control unit 1939 work together to perform sequences of micro-operations needed to perform "fetch, decode, and execute" cycles for 20 every instruction in the instruction set making up the program 1933. Each fetch, decode, and execute cycle comprises: (a) a fetch operation, which fetches or reads an instruction 1931 from a memory location 1928, 1929, 1930; (b) a decode operation in which the control unit 1939 determines which instruction 25 has been fetched; and 5552872_2 P001898_specilodge - 13 (c) an execute operation in which the control unit 1939 and/or the ALU 1940 execute the instruction. Thereafter, a further fetch, decode, and execute cycle for the next instruction may be executed. Similarly, a store cycle may be performed by which the control unit 1939 5 stores or writes a value to a memory location 1932. Each step or sub-process in the processes of Figs. 4 to 8 and 20 and 21 is associated with one or more segments of the program 1933 and is performed by the register section 1944, 1945, 1947, the ALU 1940, and the control unit 1939 in the processor 1905 working together to perform the fetch, decode, and execute cycles for every instruction in the 10 instruction set for the noted segments of the program 1933. The term "template shape" as used herein refers to a predetermined model that defines the form of a modifiable non-textual shape object with a closed outer boundary within a specified editing or viewing environment. Accordingly, the template shapes described herein are "modifiable closed-form non-textual template shapes". Many 15 template shapes consist of a single closed outer boundary, although some, such as the "can" template shape 27102 shown in Fig. 2B, include other internal lines. Some other template shapes, such as the "sun" template shape 27104 in Fig. 2B are made of more than one closed outer boundary, and are referred to herein as "disjoint template shapes". A template shape is specified by a plurality of parameters the interpretation of which is 20 defined within a model for the template shape. The plurality of parameters can include affine parameters, control parameters and line thickness parameters. The parameters defining a template shape are described in further detail below. Figs. 2A and 2B show a selection of one hundred and four MicrosoftTM Autoshapes 27001 to 27104 which are a set of template shapes. Further, Appendix A is a Table listing 25 the names of each of the MicrosoftTM AutoShapes of Figs. 2A and 2B, together with their drawing reference in Figs. 2A and 2B. 5552872_2 P001898_speci_lodge -14 The affine parameters of the template shape define modifications to the closed form non-textual template shape. Such modifications may be described by an affine transform. The modifications include scaling, rotation and offset. For scaling of sx and sy along the x- and y-axes respectively, a rotation of 0, and offsets of Ax and Ay along the x 5 and y-axes respectively, the affine transform may be represented in accordance with Equation I below: (-sinO cosO6 sx 0 Ax 0 cosO sinO 0 sy 0 Ay Control parameters define further geometrical modifications to the template shape that in general cannot be defined in terms of an affine transform of the template shape. 10 Each template shape has a defined set of N control parameters, where N is specified by the model and may be zero. Template shapes may be defined entirely by straight edges (i.e. polygons) or curves, or may comprise a mixture of straight edges and curve sections between a set of vertices. Each control parameter typically modifies a plurality of vertices, a plurality of curve sections, or both a plurality of vertices and curve sections. Control 15 parameters are typically modified by moving one or more control points. Control points are generally visible in a graphical user interface (for example as small yellow diamonds) when the shape is selected, and the user may modify positions of the control points using the mouse or other form of user input. Control points operate as shape modification points and can have a variety of different constraints applied to them, such as start and end limits 20 or be movable in x-axis or y-axis only. Control parameters are functions of the x-axis and y-axis position relative to the bounding box of the template shape of one or more control points. One control point can be the input to multiple control parameters. The effect of modifying the control points (and hence the control parameters) of a template shape is defined by the template model for the template shape. The control 25 parameters may be used for modifying the closed-form non-textual template shapes in a 5552872_2 P001898_specilodge - 15 non-affine manner. A set 300 of example template shapes 301 to 324 each with one control point is illustrated in Fig. 3. Adjacent to each template shape 301 - 324 is a black square representing the control point, the x-coordinate of which represents the value of the corresponding control parameter. Fig. 3 includes a set of rounded rectangles (i.e., 301 to 5 304), for which the control parameter modifies the radius of curvature of the rounded corners. Fig. 3 also includes trapezoids (i.e., 305 to 308), hexagons (i.e., 309 to 312) and octagons (i.e., 313 to 316) for which the control parameter defines the point of intersection of a line on the left side of the template shape with the upper horizontal section. Fig. 3 also includes 16-point seals (i.e., 317 to 320) for which the control parameter (e.g., 326) defines 10 the relative radii of the inner and outer vertices. Finally, Fig. 3 includes moon shapes (i.e., 321 to 324) for which the control parameter (e.g., 327) defines the phase of the moon shape. A typical scale used for control parameters has a maximum range of 0 to I where, for example, a horizontal parameter such as used in the hexagon 309 to 312 would be 0 on 15 the far left, and 0.5 when the modifiable point is brought into the middle. 0.5 is then the maximum value for this shape, but a control parameter which ranges all the way from left to right, such as the parallelogram 27007 in Fig. 2A would take the full set of values from 0 to 1. Fig. 4 illustrates a first approach 400 used to create an output document file 150 by 20 processing module 140. The method 400 is typically realised as a software application program stored on the HDD1910 and executable by the processor 1905. In step 410, the input image 130 is retrieved from the HDD 1910 by the processor 1905 and segmented. During segmentation, the processor 1905 examines pixel values in the input image 130 to find regions of approximately constant or in some cases approximately smoothly varying 25 colour. The identified regions may be temporarily stored in the memory 1906. The regions are then joined together based on proximity, with the output of the segmentation 5552872_2 P001898_specilodge - 16 process 410 being a set of constant colour or smooth gradient objects corresponding to regions of the input image 130. Each shape in the original document 110 is typically represented by one object for the line region and one for the fill region. Some objects may include multiple shapes connected together, such as in a flow chart. 5 In step 420, the objects produced by the segmentation 410 are analysed using page layout analysis. In this process 420, the processor 1905 identifies what type of content each object corresponds to, distinguishing content types such as text 111, table 112, images 113, and figure elements 114 and 115. The process 420 selects the objects corresponding to figure elements for further processing described below in relation to vectorization step 10 430. In some applications a full page layout analysis might not be used, but instead all objects, or almost all objects excepting a few of inappropriate sizes, may be selected to be analysed as potential figure elements. In a vectorization step 430, a shape creation method is applied by the processor 1905 to the objects selected as potential figure objects by the analysis of image content 15 step 420. This shape creation method converts the pixels of the objects selected into an approximate vector representation of zero or more (straight) line segments and zero or more curve segments, there being at least one line segment or one curve segment. These segments are approximate vector representations of the objects and are referred to herein as freeform shapes or non-template shapes. Each input shape is defined by at least one such 20 segment. The segments defining a shape may be formed as a list for ease of storage or management. Many existing vectorization techniques can be used to perform the vectorization step 430. As each shape in the original document 110 is typically represented by one object for the line region and one for the fill region, line freeforms may be created by various 25 known vectorization methods such as thinning, polygonisation of the objects followed by contour matching, or transform based techniques. Line freeforms may not be closed, but 5552872_2 P001898_speci_lodge - 17 may still be mostly similar to a template shape, many of which are closed. Open line freeforms may be processed in the same way as closed freeform shapes by linking the ends of lines to other nearby line ends. Fig. 13B shows an example open line freeform 1330 which can be processed as a 5 closed line freeform by linking the ends 1332 and 1334 of the line together, for example with a single straight segment. In general it may be possible to associate the fill object with the line object to create a freeform shape which has both a line and a fill, along with various parameters such as line thickness and line and fill colour. All the example template shapes shown in Figs. 2A and 2B are line freeforms, having a black line with a 10 given thickness and a white fill and some shadow effects where a grey fill is visible. Fig. 9A shows a freeform arrow 900 that has been scanned and vectorized. The outline of the arrow 900 has been decomposed into a linked sequence of straight line segments, such as segment 920. Each segment is defined by a start and end points, such as points 910 and 930 that define the segment 920 and all points thereon. The vectorized 15 representation of the arrow 900 can be saved by storing the start and end points in a list to allow reconstruction of the segments. The start and end points may be used as control points for potential modification of the vectorized shape, for example by a user clicking and dragging the control point via a GUI such as discussed above. An example of such modification is seen in Fig. 9B where the control point 910 has been manually moved via a 20 GUI to be a point 910', thus altering the segment 920 to be a segment 920' and thereby editing the shape 900 to give a new shape 900'. Returning to Fig. 4, in step 440, a process of forming parameterised shapes is applied by the processor 1905 to the freeform shapes (segments) created in step 430. Step 440 generates one or more parameterised shapes to represent at least one of the input 25 shapes. The parameterised shapes includes an appearance control parameter that varies the appearance of the shape between the geometry of the freeform shapes that were 5552872_2 P001898_specilodge - 18 processed and a modified geometry that has parts that are defined by the geometry of known template shapes parameterised to match the freeform shapes. The parameterised shapes may also include control parameters that correspond to the control parameters of the matched template shapes, and other parameters that may deform the geometry in other 5 ways. This process will be discussed in more detail with reference to Fig. 5 below. Figs. 14A to 14E illustrate such a parameterised shape in a number of configurations of the appearance control parameter that might be created at step 440 based on the input freeform shape shown by 900 in Fig. 9A. Fig. 14A shows a first configuration 10 1410 of the parameterised shape configured such that the appearance is given by the input freeform shape, and Fig. 14E shows a last or final configuration 1450 of the parameterised shape such that the appearance is given by the template arrow shape. Figs. 14B, 14C and 14D show intermediate configurations 1420, 1430 and 1440 respectively. The appearance parameter is shown by the star 1413(1), 1413(2), 1413(3), 1413(4) and 1413(5) as the 15 appearance is deformed through the configurations 1410, 1420, 1430, 1440, and 1450. The arrow template shape on which the parameterised shape was based has two control parameters, illustrated by the control parameters 1452 and 1453 in Fig. 14E, and these have been generalised to work in all appearances of the parameterised shape. For example, in the freeform appearance configuration 1410 of Fig. 14A, these control parameters are 20 1411 and 1412. Following step 440, at step 450, the output document file 150 is created by the processor 1905. This document includes a representation of the template shapes and freeform shapes from step 430 with control points (the representative form) as produced by step 440. This output file 150 may be written in the MicrosoftTM (OOXML) Office Open 25 XML format for MicrosoftTM WordTM or PowerPointTM, or may alternatively for example be written in the OpenDocument file format (ODF). Various other file formats may also 5552872_2 P001898_specilodge - 19 be suitable. The output document may also include the results of analysis of other types of content such as text. Figs. 18A to 18C illustrate a second parameterised shape that describes a hand drawn flowchart based on an input freeform shape 1810 seen in Fig. 18A that includes a 5 number of freeforms including freeform shapes and freeform connectors. These freeform shapes have been matched to four template rectangle shapes, three template connectors and an arrowhead. The corresponding template appearance, given by the representation 1830 of Fig. 18C includes matched template geometries of the input freeform shapes. Note that in addition to the shapes now being rectangles, shapes 1832 and 1833 are now aligned the 10 horizontal direction, and have the same width. Also the connectors now connect to the midpoints of the shape edges. A parameterised shape is generated with an appearance control point 1811, seen in Fig. 18A at position 1811(1). The position of the appearance control point 1811 in a GUI can be modified to adjust the appearance between the freeform shape geometry 1810 and the template shape geometry 1830. For example when the 15 appearance control parameter 1811 is set about half way between the freeformm' and 'template' endpoints to a position 1811(2), the geometry of the parameterised shape is given by an intermediate representation 1820 seen in Fig. 18B. The template appearance 1830 of Fig. 18C is obtained when the appearance control parameter 1811 is moved to a position 1811(3) in the GUI. As such, a single appearance control point 1811 may control 20 modification of a plurality of shapes and their combination, in this case as a flowchart. Fig. 5 illustrates an exemplary process 500 used to form parameterised shapes of step 440 in Fig. 4. In step 510, a next unprocessed freeform shape is selected. In step 520 a parameterised shape is formed by the processor 1905 using a process that is described with reference to Fig. 6. In step 530 a test is made by the processor 1905 to determine if 25 there are more unprocessed freeform shapes. If there are none, then process 500 is finished. Otherwise, the next freeform shape is selected at step 510. 5552872_2 P001898_specijlodge - 20 Fig. 6 illustrates a preferred process 600 used to form a parameterised shape from a freeform shape as discussed in step 520. At a receiving step 610, the input freeform shape comprising at least one line segment and/or at least one curve segments is received from the freeform shape generating step 430. The line and curve segments may be used to 5 classify the freeform shape as one or more configured candidate templates, at step 620. In the classification step 620, the processor 1905 selects one or more candidate template shapes, which may be temporarily stored in the memory 1906, to which the input freeform shape is similar. One method which can be used to perform the classification step 620 is a machine learning classifier based on inputs derived from the line and curve segments of the 10 freeform shape. A large number of training template shapes can be created with various control parameter values, with the area of the template shape being normalised to one (unit squared), the (1,1) central moment normalised to zero and the (2,0) and (0,2) central moments normalised to be equal. A sampling of the perpendicular distance from the unit circle can then be Fourier transformed to produce the feature training data for a machine 15 classifier, such as a Support Vector Machine (SVM). This will produce a number of SVMs, one for each pair of template training shapes, which are used to decide whether the input shape is one template shape or another. The normalising procedure can then be applied to the input freeform shape and all of the SVMs applied to the result. This will produce a score for each template shape, and the template shapes with the highest scores 20 can be selected as being candidate templates. Alternatively, if there is only a small number of template shapes, then all of them can be selected as candidate templates at classification step 620. At the classification step 620, a set of parameters is calculated for the candidate template shapes that would configure the template to closely match the input freeform 25 shape. The parameters of the template shapes, as described above, include scaling and rotation, as well as any control parameters particular to the template shape. One method 5552872_2 P001898_spedlodge -21 for calculating the control parameters of a template shape to closely match those of a freeform shape is to affine normalise the freeform shape so that the area is unity, the (1,1) central moment is zero, and the (2,0) and (0,2) central moments are equal. The Zernike moments can then be calculated for the input freeform shape and a template shape with an 5 initial set of control parameters. Multi-dimensional functional minimisation, such as the downhill simplex method, can then be used to find the set of control parameters that minimise the distance between the Zernike moments of the freeform shape and the Zernike moments of the template shape. One method for calculating the scale estimation for an input shape is to use the (2,0) and (0,2) central moments for the input freeform shape and 10 compare to the (2,0) and (0,2) central moments of the template shape (with control parameters applied). One method for calculating the rotation estimate is to divide the template shape (with scaling and control parameters applied) and the input freeform shape into a number of equally sized wedges (e.g. 64) centred on each shape. The area of the shape inside each wedge can then be correlated between the two shapes, with the peak in 15 the correlation function giving the rotation angle estimate. In step 630, a similarity measure, or match score, is calculated by the processor 1905 to evaluate how well each template shape matches the freeform shape when configured according to the parameters found in step 620. Various techniques may be used to calculate the similarity measure, such as a normalised area based mismatch, Hausdorff 20 distance, or earth-movers distance. In general, for the technique to be useful for step 630, the difference between the union and the intersection of the template shape and the input freeform shape should be penalised. The similarity measure may be calculated such that a higher similarity measure or score indicates a closer visual match than a lower measure, and is used to select a template shape, for example from the shapes of Figs. 2 and 3. 25 In step 640, a parameterized shape is created based on the input freeform shape and the template shape with the highest match score as calculated and selected in step 630, 5552872_2 P001898_speci_lodge - 22 referred to as the selected candidate template shape. The parameterized shape created is parameterized by both the control points of the selected template shape and an additional appearance control point that can vary the appearance of the parameterized shape between the input freeform shape and the matching configuration of the selected template candidate. 5 The appearance control parameter operates to modify the appearance of the parameterized shape over a range of appearances with the freeform shape appearance at one end and the selected template shape appearance at the other. The range may operate as a continuous variation or as a predetermined number of steps between the freeform shape and the selected template candidate. 10 In step 650, the parameterized shape is converted to a form suitable for output in the desired output document format. The output document generally has features suitable to represent control points, and can represent the location of points, segments and curves as a function of the location of the control points. These functions may be built up of primitives such as addition, multiplication and various transcendental functions (e.g. 15 cosine, sine). Fig. 7 illustrates a process 700 used in step 640 to create a parameterized shape in Fig. 6. In a grouping step 710, the freeform segments are grouped to match the segments of the matching configuration of the selected candidate template from step 630. Grouping freeform segments to match the template segments may use a method specific to the 20 template shape. A general method may be to find the closest freeform point to each of the endpoints of the segments of the selected candidate template. The freeform points found this way can then be used to divide the freeform segments into two or more groups, with the points found being the start or end point of a particular group of segments. The freeform segment groups are then processed in turn. For example, Fig. IOC depicts 25 matching of an arrow template shape 1010 with a similar freeform 1020. 1030 and 1032 are adjacent points on the template shape 1010. 1031 is the closest point on the freeform 5552872_2 P001898_specilodge - 23 1020 to the template shape point 1030, and 1033 is the closest point on the freeform 1020 to the template shape point 1032. There are no other points on the freeform between 1033 and 1031 heading clockwise around the freeform 1020 (i.e. right and down from 1033) that are closest to any of the other points on the template shape. Therefore, the set of 5 freeform segments between 1033 and 1031 form a group of freeform segments for later processing. At step 720 the next freeform segment group is selected. The order in which the groups are processed is not important. The grouped freeform segment is then parameterized both by the control points of the template shape and an appearance control 10 point at step 730, in accordance with a method that will be described below with reference to Fig. 8. The appearance control point operates to vary the appearance of the freeform from the initial appearance at a freeformm' endpoint to an appearance similar or identical to the template shape at the 'template' endpoint. At decision step 740, if there are more freeform segment groups then flow returns to step 720. Otherwise control proceeds to step 15 750, where the location of the appearance control point is determined. Such location may be specific to each type of template shape, but in general some rules and heuristics may be applied. From an end user's point of view, it is useful that the paths of the control points do not overlap for non-degenerate bounding boxes of shapes. Many template shapes have control points that travel along a bounding box edge. If there is a corner of the bounding 20 box that the control points of the template shape can never reach, then that corner is a good candidate to place the appearance control point. The appearance control point can then use the corner of the bounding box as one end point of movement and the midpoint of an unused edge as the other end point of movement. If there are no unused corners of the template shape bounding box, then the appearance control point can be offset to either 25 slightly inside or outside the shape bounding box. 5552872_2 P001898 specilodge -24 Fig. 8 illustrates an exemplary process 800 to parameterize grouped freeform segments, useful in step 730 of Fig. 7. In step 810 the endpoints of the freeform group are parameterized with respect to both the control points of the template shape and the appearance control point. The endpoints of the freeform group may be parameterized first 5 with respect to the control points of the template shape. This may use a method specific to the template shape, but in many cases, the control points of the template shape can be thought of as dividing a bounding box of the template shape into two or more regions, where the control points of the template shape change the relative sizes of those regions. These regions can be mapped to the bounding box of the freeform shape. The end points 10 of the freeform group can be parameterized using an affine transform depending on the region in which the point is contained. As the control parameters of the parameterised shape are varied, the freeform point stays in the same location within the corresponding freeform region, however the geometry of the freeform region is altered according to the control parameter. Once the end points of the freeform group have been parameterized by 15 the template control points, they can additionally be parameterized by the appearance control point. One way to do this is to interpolate (linearly or otherwise) from the end points of the freeform group to the respective segment endpoints on the template shape. This ensures that when the appearance control point is moved to select an appearance more like a template shape, there is known to be at least one freeform point moving to the 20 location of the endpoints of the template shape segment. Linear interpolation can be expressed using the formula (2) below. The vector x=(x,y) can be expressed in term of vectors xod and Xnew, and the parameter t, where t can vary between 0 and 1 inclusive. Note that both xold and Xnew may also be parameterizations. For example, the freeform group endpoints can be represented using 25 formula (2) below, where x 0 ld is the parameterization by the template control points, and 5552872_2 P001898_specijlodge - 25 xnew is the template endpoint (which may also be parameterized by the template control points). The appearance control point then controls the value of t between 0 and 1. X Xold tXew Xold (2) Y _Yold Ynew _ LYold . A check is then made at decision step 820 as to whether there are more points in the 5 freeform groups. For example if the freeform group has one segment, then that freeform group has no points other than the end points. A freeform group with two segments has three points, and so one point which is not an endpoint. At decision step 820 if there are more points in the freeform segment group, then flow proceeds to step 830. Otherwise process 800 is finished. At step 830 the next point in the freeform segment group is 10 selected. The order in which the points of a segment of a freeform group are processed is not important. At step 840, the closest point on the template segment is found to the freeform segment group point. This is generally the intersection of the line through the freeform segment group point which intersects the template segment line at a right angle. However, as the template segment is not an infinitely long line, where the intersection line 15 intersects the template segment line outside the template segment itself, the closest point is instead the nearest endpoint of the template segment. At parameterization step 850, the closest point on the template segment found at step 840 is parameterized with respect to the control points of the template shape. This can be achieved by parameterizing the closest point on the template segment in the same way that the freeform segment group 20 endpoints were parameterized with respect to the control points of the template shape. As such, the closest point on the template segment can be parameterized generally using an affine transform based on the control points of the template shape. At parameterizing step 860, the freeform segment group point is parameterized in a way similar to the freeform segment group endpoints in step 810. The freeform segment group point may be 25 parameterized first with respect to the control points of the template shape, as described 5552872_2 P001898_spedlodge - 26 above for the freeform segment group endpoints in step 810. Next, the freeform segment group point may be parameterized by the appearance control point. One way to do this is to interpolate (linearly or otherwise) from the freeform segment group point to the parameterized closest point on the template segment calculated in step 850. It may be 5 possible to simplify the parameterization of the freeform segment group point in step 870. This step depends on the geometry of the template shape, such as template segments that are not parameterized by the control points of the template shape, or template segments that are parameterized in only the x-axis or y-axis. It may be the case that the x-axis and y axis components of a point are calculated separately. In this case it may be possible to 10 simplify one of the axis components by not having to parameterize that axis component using the appearance control point. For example, consider input freeform 900 from Fig. 9A and point 910. As the template shape segment which matches the freeform segment group that point 910 belongs to only moves in the y-axis (i.e. down and not sideways as seen in Fig. 9B), only the y-axis component of point 910 needs to be parameterized by the 15 appearance control point. Step 870 is not strictly necessary to produce a parameterized shape, but does reduce the complexity (and hence, the output file size) of the functions required to parameterize the input freeform. An example of some of the steps in processes 600, 700 and 800 are now described with reference to Figs. 9 to 13. Fig. 9A shows an example input freeform 900 with 20 segment ends highlighted. An example segment is 920 with ends 910 and 930. Freeform 900 could be the input to step 600, where one of the classified template shapes calculated could be the left arrow template shape 27028 from Fig. 2. The left arrow template shape 27028 could also be the template shape with the highest match score out of the template shapes selected in classification step 620. 25 Fig. 10A shows the example input freeform 1020 with the closest template match 1010 which, in this case, is akin to the connector template shape 27028 seen in Fig. 2. 5552872_2 P001898_specijodge - 27 The template shape 27028 is a left arrow and may be processed by step 630 to calculate the scaling, rotation and control point parameters. The control points of the template shape 1010 are circled in Fig. 10A, examples of which are shown as 1001 and 1002. These control points alter the width of the body compared to the arrowhead and the length of the 5 arrowhead respectively. Fig. 10B shows the template shape as 1010 with control points circled and shown as 1013 and 1014. Also shown are the regions of the template shape shown by dividing lines 1015, 1016, 1017 and 1018. A user moving the control points 1013 and 1014 via a GUI reproduced on the display 1914 will change the relative sizes of the regions shown. The axis of the bounding box of the template shape 1010 are shown as 10 1011 and 1012. Fig. 11 shows a convex hull 1110 of an input freeform shape 1120. A convex hull of a set of points is a convex polygon that contains all the points in the set. A simple method to construct a convex hull for a set of points in two dimensions is known as the "Jarvis march". An initial point known to be on the convex hull is selected, for example 15 the leftmost point in the set. The next point selected to form the convex hull is one where all the other points in the set are to the right of the line formed between the initial point and the new point. The process is repeated with the new point as the initial point until the leftmost point is found again. The points where the convex hull I110 diverges from the input freeform 1120 are shown circled as 1111, 1112, 1113, 1114 and 1115. For the two 20 largest divergences, the points on the input freeform 1120 which are furthest from the convex hull 1110 are shown circled as 1121 and 1122. The convex hull divergence points 1I11, 1112, 1113 and 1115, as well as points furthest from the convex hull 1121 and 1122 can be used to group the freeform segments as described in step 710. The point 1114 can be found by noting that the matched template shape is a left arrow and finding the leftmost 25 point of the freeform 1120, after correcting for the rotation of the matched template shape. 5552872_2 P001898_specijlodge - 28 The endpoints of the freeform segment groups 1111, 1112, 1113, 1114 and 1115 can then be parameterized as described in step 810. Fig. 12 shows an example freeform segment group 1210 and an example template segment 1220. The template segment 1220 has endpoints 1221 and 1222. The freeform 5 segment group 1210 has endpoints 1211 and 1215, along with example other points shown as 1212 and 1214. As described by step 810, the endpoints of segment group 1210 are parameterized to the endpoints of the template segment 1220. So point 1211 will be parameterized to point 1221, and point 1215 will be parameterized to point 1222. The closest point on the template segment 1220 to freeform point 1212 is shown as point 1213. 10 Point 1213 is found as described in step 840. The closest point on the template segment 1220 to point 1214 is endpoint 1222, as described in step 840, the closest perpendicular point to line 1220 lies outside the segment endpoints. The point 1213 can then be parameterized as described in step 850 by using an affine transform based on the template shape's control points. In the example template shape 1010 of Fig. lOB, the regions which 15 determine the affine transform used are shown by 1015 to 1018. Fig. 13A shows the regions of the example template shape 1010 overlaid onto the example input freeform shape 1320, with the region boundaries shown by 1323, 1324, 1325 and 1326. The control points of the template shape 1010 are shown circled as 1321 and 1322. The region boundaries 1323, 1324, 1325 and 1326 allow the points of freeform 20 shape 1320 to be parameterized by the control points of template shape 1010, as described in step 860 of Fig. 8. After the freeform points of shape 1320 are parameterized, using for example an affine transform, they can be parameterized by the closest points to the template segments as calculated at step 840. Fig. 13B shows an example freeform input 1330 that is not closed. However, the freeform 1330 can be processed in the same way as 25 example input freeform 900 by joining endpoints 1332 and 1334 to create a closed input freeform shape. 5552872_2 P001898_speci-lodge - 29 Fig. 14A shows the output parameterized shape 1410 for the input freeform shape 900 in Fig. 9A. The shape 1410 has control points 1411 and 1412 corresponding to those of the corresponding matched template shape (e.g. 20708).An appearance control point 1413 is also shown positioned at the 'freeform' endpoint of a range in which the 5 appearance control point 1413 is movable within a GUI. The range over which the control point 1411 can move is the entire top edge of a bounding box 1414 of the shape 1410, as indicated by the arrow 1415, and the range that the control point 1412 can move is the upper half of the right edge of the bounding box 1414 of the shape 1410, as indicated by the arrow 1416. Therefore, in this example, a suitable place to locate the appearance 10 control point 1413 is the lower half of the left edge of the bounding box 1414 of shape 1410, as described by step 870. In the case of appearance control point 1413, the 'freeform' endpoint is the middle of the left edge, as seen in Fig. 14A and designated 1413(1), and the 'template' endpoint is the bottom of the left edge of the bounding box of shape 1410, discussed below. The appearance of the output parameterized shape 1410 is 15 similar to the input shape 900. In Fig. 14B, the appearance control point 1413 has been moved in the GUI to a second position 1413(2) about a quarter of the way from the 'freeform' endpoint towards the 'template' endpoint, this having the effect of altering the parameterised shape 1410 to the appearance 1420. The same output parameterized shape is shown as 1430 in Fig. 14C, 20 where the appearance control point 1413 has been moved to a position 1413(3) about half way between the freeformm' and 'template' endpoints. The same output parameterized shape is shown as 1440 in Fig. 14D, where the appearance control point 1413 has been moved to a position 1413(4) about three quarters of the way to the 'template' endpoint. The same output parameterized shape is shown as 1450 in Fig. 14E, where the appearance 25 control point 1413 is in a position 1413(5) at the 'template' endpoint. Note that the two template control points 1411 and 1412 have not been moved. Note that in each instance of 5552872_2 P001898_speci_lodge -30 Fig. 14A- 14E, the shape control points 1411 and 1412 are unmoved, and remain at corresponding positions in each of the appearances 1410-1450. Also note that this example was constructed with the template shape horizontal as shown in 1450, rather than at an angle as in 1010 in Fig. 10. 5 Figs. 15A and 15B shows various examples of output parameterized shapes 1501 to 1506 and 1511 to 1516 for the input freeform shape 900 in different configurations with modified template control points. In each instance, an appearance control point 1599 is shown and remains unmoved and fixed at the same location in each appearance 1501-1506, 1511-1516. 10 In Fig. 15A, shapes 1501 to 1506 show the output parameterized shape where a first template control point 1500 has been set to various positions 1501(1) - 1501(6). This, as seen has the effect of varying the length of the head of the arrow. In Fig. 15A a second template control point 1510 remains unmoved and fixed. Fig. 15B shows shapes 1511 to 1516 of the same output parameterized shape 15 where the template control point 1510 has been set to various positions, 1510(1) - 1510(6). This has the effect of varying the thickness of the shaft of the arrow. In Fig. 15B, the template control point 1500 remains unmoved and fixed, and substantially in the position 1500(3) as indicated. Figs. 16A and 16B shows examples of the output parameterized shapes 1601 to 20 1606 and 1611 to 1616 for the input freeform shape 900 in different configurations with modified template control points 1600 and 1610. The template control points 1600 and 1610 are in similar positions to those shown as 1500 and 1510 in the examples of Figs. 15A and 15B respectively, although indications thereof have been omitted for clarity. In each of the examples of Figs. 16A and 16B however, the appearance control point 1699 25 has been set about half way between the 'freeform' and 'template' endpoints. 5552872_2 P001898.specilodge -31 Fig. 17A shows an example input of a shape 1710 that may not be a known template shape. The shape 1710 has been processed to produce a parameterized shape with template control points 1712 and 1713 and an appearance control point 1711. The input shape is shown as a parameterized shape 1720 in Fig. 17B where the appearance control 5 point has been moved to a position 1711(2) about half way between the 'freeform' endpoint and the 'template' endpoint. The input shape is also shown as a parameterized shape 1730 in Fig. 17C where the appearance control point has been moved to a position 1711(3) corresponding to the 'template' endpoint. Note that the shape 1710 and the appearances 1720 and 1730 thereof, behaves in a similar way to a new template shape with 10 three control points, with the two control points from the template shape and the third being the appearance control point. Returning to Figs. 18A - 18C, Fig. 18A show an example input 1810 of several shapes and connectors, representative of a hand drawn flowchart. The input 1810 has been processed to produce parameterized shapes with an appearance control point 1811. The 15 same parameterized shapes are shown modified in Fig. 18B as a flowchart 1820 with the appearance control point 1811 shifted to a position 1811(2) about half way between the freeformm' and 'template' endpoints. The same parameterized shapes are seen in Fig. 18C as a flowchart 1830 where the appearance control point 1811 is shifted to the 'template' endpoint position 1811(3). Note that in addition to the shapes in Fig. 18C being now 20 modified to rectangle outline, other properties of shapes 1832 and 1833 have been varied, in this case so that the shapes are aligned in the horizontal direction, have the same width and have been adjusted to locate connectors to the midpoint of the shape edges. Variation of properties such as the horizontal alignment, shape width, shape height, shape length and connection location may be changed in a manner as described above. It is also possible to 25 configure the appearance control point 1811 to not vary the shape properties, or only a subset of the various properties. In the example of Figs. 18A - 18C, the input freeform 5552872_2 P001898_speci_lodge - 32 shape comprises a plurality of shapes each of which is matched to a template shape. From the combination of those template shapes is formed the modifiable shape controllable by a single appearance control point. Where at least one template shape comprises a connector, an endpoint of that connector is desirably set to a connector point of another template 5 shape, for example in the middle, top, bottom or side of a rectangle as illustrated. The net effect of the arrangements presently disclosed is that a freeform input shape, such as the arrow 1410 of Fig. 14A can be interpreted as being akin to a most similar template shape 27028 of Fig. 2A, and then parameterized with control points 1411 and 1412 corresponding to those of the matched template shape 27028. Note in this 10 example the freeform arrow 1410 has a relatively fat head whereas the template shape 27028 has a relatively narrow head. Manipulation of the control point 1411 in the horizontal direction provides for user variation of the thickness of the head (see 1501-1506 of Fig. 15), while user manipulation of the control point 1412 in the vertical direction in a GUI provides for variation of the thickness of the arrow shaft (see 1511-1516 of Fig. 15). 15 See also Fig. 16 for examples of where both control points 1411 and 1412 are manipulated. According to the present disclosure, the further control point 1413 is provided to be user manipulable to vary the overall representation or appearance of the arrow from the freeform version of Fig. 14A, to that of Fig. 14E, being that most akin to the matching 20 template shape 27028. Further manipulation of the further control point, now 1413(5) toward the position of 1413(1) can return the template-like representation of the arrow to that of freeform original of Fig. 14A. As such, not only can a freeform shape be parameterized and matched to an editable template shape, but such editable shape may further then be user manipulable within a GUI between the freeform and template shape 25 appearances. 5552872_2 P001898_speci_lodge - 33 One specific advantage of this approach is that, where the user is not satisfied with the selection of the matching template shape, the freeform can be returned to its original shape and potentially re-parameterized according to another template shape, for example as selected by the user. 5 A second approach 2000 of processing the input document image I10 for generating an editable document output file 150 is now described with reference to Fig. 20. Fig. 20 shows a flowchart of a method 2000 that is again preferably realised by software executable by the processor 1905 of the computer system 1900 and operatable to modify an existing document to replace any freeform shapes with either template shapes or non 10 template shapes with control points. At step 2010, the freeform shapes are extracted from the document. Step 2010 is essentially equivalent to steps 410, 420 and 430 of Fig. 4. At step 2020, the input freeform shapes are converted to either template shapes, or non template shapes with control points. This step is the same as step 440 described above with reference to Fig. 4. In step 2030 the input freeform shapes are replaced in the 15 document with parameterized shapes with an appearance control point, this being an alternative to step 450 of Fig. 4. Alternatively, this approach can be used to copy template shapes onto the clipboard associated with the operating system of the computer 1901 so that the shapes can be cut and pasted into another program with a different set of template shapes. As such, the input (freeform) shape may indeed be a template shape, but not a 20 predefined template shape that is recognised by the present software application 1933, but which is for example a template in another, unrecognised, software application. In operation, this input (unrecognised template) shape is treated as a freeform, and by which the method 440 parameterizes the input shape with template control points based on those contained in a known template shape (e.g. Fig. 2), and by which the appearance control 25 point is manipulable to change the appearance from the unrecognised template shape to the known template shape. 5552872_2 P001 898_specilodge - 34 A third approach is now described, which is similar to the first approach 400, except a global appearance control point is created. The third approach is performed according to the process 2100 described below with reference to Fig. 21. In parameterization step 2110, parameterized shapes are created for a set of input freeforms. 5 The freeforms may be created by steps 410 to 430 of Fig. 4, or from step 2010 from Fig. 20. The set of input freeforms may be all input freeforms, or a subset that are conceptually related to each other. As an example of a set of input freeforms which are related to each other, consider shapes shown as 1810 from Fig. 18A to 18C. An earlier process may have split input 1810 into four rectangle like shapes and three connector shapes. Each of these 10 individual freeform shapes may be processed separately to create parameterized shapes, each with an individual appearance control point. The template shape for the connectors could be a straight line between the endpoints of the input freeform. In replacement step 2120, the individual appearance control points for the set of parameterized shapes created in step 2110 are replaced by a global appearance control 15 point. The global appearance control point could be placed on the edge of the bounding box of the set of parameterized shapes. One way to replace the individual control points is to note that each appearance control point is created in a similar way, and moving the control point adjusts a parameter in the definition of each parameterized shape. This parameter can be updated to refer to the global appearance control point rather than the 20 appearance control point of the individual shape. Alternatively the individual appearance control points may be maintained and a global appearance control point may be added. When the global appearance control point is added there are at least two possible modes of interaction between the global and individual shape appearance control points. The first interaction method allows the global appearance control point to configure all of the 25 individual appearance control points. A user may adjust the global appearance control point before making adjustments to any of the individual appearance control points. If the 5552872_2 P001898_specijodge - 35 global appearance control point is then adjusted the individual appearance control point will be reset according to the value of the global control point. Alternatively the individual appearance control point may be configured to provide a relative adjustment to the global appearance control point. The individual shape may be set, using the appearance control 5 point, to a value 10% lower than the global appearance control point. Any changes to the global appearance control point will be reflected for the individual appearance control point while maintaining the relative value. In a rationalization step 2130, which need only optionally be performed, the configurations of the template shapes used to generate the parameterised shape are adjusted 10 based on a set of rules and the parameterised shape are altered accordingly. These rules may include aligning shapes and setting widths or heights to be equal, or making the endpoints of connectors the midpoints of the connecting shape. An example of these adjustments are shown in Fig. 18C, where the appearance control point 1831 is at the 'template' endpoint, and the rectangles 1832 and 1833 have been vertically aligned and set 15 to have the same width. The connectors shown in 1830 have also been set so that they connect to the midpoint of the connecting shape. In an output step 2140, the set of parameterized shapes are output to the document as described in step 450 of Fig. 4. In some document types, it may not be possible to output multiple parameterized shapes which can be controlled by a single appearance 20 control point. In this case, merging the set of parameterized shapes into one shape is a possible solution to this problem. Also note that process 2100 can be adapted to process input shapes such as pie or bar charts, and to add control points to control the percentage of each pie wedge or the height of the bars in a bar chart. An appearance control point may also be added that performs a rationalization step to the chart. 25 INDUSTRIAL APPLICABILITY 5552872_2 P001898_speci_lodge -36 The arrangements described are applicable to the computer and data processing industries, and particularly for the generation of editable documents from bitmap images. The foregoing describes only some embodiments of the present invention, and modifications and/or changes can be made thereto without departing from the scope and 5 spirit of the invention, the embodiments being illustrative and not restrictive. (Australia Only) In the context of this specification, the word "comprising" means "including principally but not necessarily solely" or "having" or "including", and not "consisting only of". Variations of the word "comprising", such as "comprise" and "comprises" have correspondingly varied meanings. 5552872_2 P001898_speci_lodge - 37 Appendix A TABLE 1 SHAPE NAME REFERENCE Rectangle 27001 Round Rectangle 27002 Ellipse 27003 Diamond 27004 Isosceles Triangle 27005 Right Triangle 27006 Parallelogram 27007 Trapezoid 27008 Hexagon 27009 Octagon 27010 Plus Sign 27011 Star 27012 Arrow 27013 Thick Arrow 27014 Home Plate 27015 Balloon 27016 Seal 27017 Plaque 27018 Chevron 27019 Pentagon 27020 Seal8 27021 Seal 16 27022 Seal32 27023 Wedge Rectangle Callout 27024 Wedge Rrect Callout 27025 Wedge Ellipse Callout 27026 Wave 27027 Left Arrow 27028 5552872_2 P001898_speci_lodge - 38 Down Arrow 27029 Up Arrow 27030 Left Right Arrow 27031 Up Down Arrow 27032 Irregularseal 1 27033 Irregularseal2 27034 Lightning Bolt 27035 Heart 27036 Quad Arrow 27037 Left Arrow Callout 27038 Right Arrow Callout 27039 Up Arrow Callout 27040 Down Arrow Callout 27041 Left Right Arrow Callout 27042 Up Down Arrow Callout 27043 Quad Arrow Callout 27044 Left Up Arrow 27045 Bent Up Arrow 27046 Bent Arrow 27047 Seal24 27048 Notched Right Arrow 27049 Block Arc 27050 Circular Arrow 27051 U Turn Arrow 27052 Flow Chart Process 27053 Flow Chart Decision 27054 Flow Chart Input Output 27055 Flow Chart Document 27056 Flow Chart Terminator 27057 Flow Chart Preparation 27058 Flow Chart Manual Input 27059 Flow Chart Manual Operation 27060 Flow Chart Connector 27061 Flow Chart Punched Card 27062 5552872_2 P001898_speci lodge -39 Flow Chart Punched Tape 27063 Flow Chart Extract 27064 Flow Chart Merge 27065 Flow Chart Online Storage 27066 Flow Chart Magnetic Tape 27067 Flow Chart Display 27068 Flow Chart Delay 27069 Flow Chart Alternate Process 27070 Flow Chart Off Page Connector 27071 Left Right Up Arrow 27072 Moon 27073 Seal4 27074 Double Wave 27075 Cube 27076 Can 27077 Donut 27078 Ribbon 27079 Ribbon2 27080 No Smoking 27081 Folded Corner 27082 Bevel 27083 Striped Right Arrow 27084 Vertical Scroll 27085 Horizontal Scroll 27086 Curved Right Arrow 27087 Curved Left Arrow 27088 Curved Up Arrow 27089 Curved Down Arrow 27090 Cloud Callout 27091 Ellipse Ribbon 27092 Ellipse Ribbon 2 27093 Flow Chart Predefined Process 27094 Flow Chart Internal Storage 27095 Flow Chart Multidocument 27096 5552872_2 P001898_specijodge - 40 Flow Chart Summing Junction 27097 Flow Chart Or 27098 Flow Chart Collate 27099 Flow Chart Sort 27100 Flow Chart Offline Storage 27101 Flow Chart Magnetic Disk 27102 Flow Chart Magnetic Drum 27103 Sun 27104 5552872_2 P001898_specilodge

Claims (20)

1. A method of creating a modifiable shape in an electronic document from a freeform input shape, the method comprising the steps of: 5 identifying the freeform shape as one of a plurality of predefined template shapes; creating a modifiable shape having at least one shape modification point defined by the identified template shape, the at least one shape modification point being positioned to configure the modifiable shape according to the freeform shape; creating a control point for the modifiable shape the control point being 10 manipulable when displayed on a display device to vary an appearance of the modifiable shape between the freeform shape and the identified template shape; and storing the modifiable shape and the control point in the electronic document.

2. A method according to claim I further comprising the steps of: 15 determining a parameter for the least one shape modification point to configure the freeform shape based on the identified template shape; and positioning the at least one shape modification point using the determined parameter. 20

3. A method according to claim 1 further comprising the step of: creating a vectorized representation of the freeform shape comprising a plurality of linked line segments wherein the appearance of the modifiable shape is varied between the freeform shape and the selected template shape by moving end points of the linked line segments. 25 5552872_2 P001898_speci lodge -42

4. A method according to claim I wherein the electronic document is stored in a single file.

5. A method according to claim I wherein the determining of parameters comprises 5 interpolating from end points of the linked line segments of the freeform shape to corresponding segment endpoints of the selected template shape.

6. A method according to claim 5 wherein the interpolating comprises linearly interpolating parameterized vectors extending between control points of the selected 10 template shape and in which a parameterizing value is varied by manipulation of the control point of the modifiable shape.

7. A method according to claim 2 wherein the determining of parameters comprises assigning end points of the linked line segments of the freeform shape to 15 corresponding segment endpoints of the selected template shape and parameterizing an intermediate points of the linked line segments using an affine transform based on control points of the selected template shape,.

8. A method according to claim I wherein the control point of the modifiable shape is 20 manipulable to modify a subset of a plurality of properties of a modifiable shape.

9. A method according to claim 8 wherein the properties of the modifiable shape comprises shape outline, shape length, shape width, shape height, shape alignment, and shape connector location. 25 5552872_2 P001898_speci_lodge - 43

10. A method according to claim 1 wherein the input shape comprises template shape not being one of the predefined template shapes.

11. A method according to claim I wherein the input freeform shape comprises a 5 plurality of shapes each of which is matched to a template shape and from which is formed the modifiable shape controllable by a single control point.

12. A method according to clam I wherein the template shape comprises a connector. 10

13. A method according to claim 11 wherein at least one template shape comprises a connector, and at least one endpoint of the connector is set to a connector point of another template shape.

14. A computer readable storage medium having recorded thereon a program 15 executable by computer apparatus to create a modifiable shape in an electronic document from a freeform input shape, the program comprising: code for identifying the freeform shape as one of a plurality of predefined template shapes; code for creating a modifiable shape having at least one shape modification point 20 defined by the identified template shape, the at least one shape modification point being positioned to configure the modifiable shape according to the freeform shape; code for creating a control point for the modifiable shape the control point being manipulable when displayed on a display device to vary an appearance of the modifiable shape between the freeform shape and the identified template shape; and 25 code for storing the modifiable shape and the control point in the electronic document. 5552872_2 P001898_specilodge - 44

15. A computer readable storage medium according to claim 14 wherein the program further comprises: code for determining a parameter for the least one shape modification point to 5 configure the freeform shape based on the identified template shape; and code for positioning the at least one shape modification point using the determined parameter.

16. A computer readable storage medium according to claim 14 wherein the program 10 further comprises: code for creating a vectorized representation of the freeform shape comprising a plurality of linked line segments wherein the appearance of the modifiable shape is varied between the freeform shape and the selected template shape by moving end points of the linked line segments, and. 15 wherein the electronic document is stored in a single file.

17. Computer apparatus comprising a processor and a memory, the memory having stored therein a program executable by the processor to create a modifiable shape in an electronic document from a freeform input shape, the program comprising: 20 code for identifying the freeform shape as one of a plurality of predefined template shapes; code for creating a modifiable shape having at least one shape modification point defined by the identified template shape, the at least one shape modification point being positioned to configure the modifiable shape according to the freeform shape; 5552872_2 P001898_speci-lodge - 45 code for creating a control point for the modifiable shape the control point being manipulable when displayed on a display device to vary an appearance of the modifiable shape between the freeform shape and the identified template shape; and code for storing the modifiable shape and the control point in the electronic 5 document.

18. Computer apparatus according to claim 17, wherein the program further comprises: code for determining a parameter for the least one shape modification point to configure the freeform shape based on the identified template shape; and 10 code for positioning the at least one shape modification point using the determined parameter.

19. Computer apparatus according to claim 18 wherein the program further comprises: code for creating a vectorized representation of the freeform shape comprising a 15 plurality of linked line segments wherein the appearance of the modifiable shape is varied between the freeform shape and the selected template shape by moving end points of the linked line segments, and. wherein the electronic document is stored in a single file.

20 20. A method of creating a modifiable shape in an electronic document from a freeform input shape, said method being substantially as described herein with reference to any one of the embodiments as that embodiment is illustrated in the drawings. Dated this 30th day of August 2011 25 CANON KABUSHIKI KAISHA Patent Attorneys for the Applicant Spruson&Ferguson 5552872_2 P001898_specilodge