JP6716409B2 - Information processing equipment - Google Patents

Description

The present invention relates to a technique of displaying a handwritten image on a shared screen.

There is a technology for displaying handwritten images on a shared screen. For example, in Patent Document 1, when the handwriting input information input at the terminal itself and the handwriting input information transmitted from another terminal are displayed on the same display surface (that is, a shared screen), the handwriting input information is transmitted. There is disclosed a technique of displaying in a display color associated with a terminal that has been used.

JP, 2010-157098, A

In the technique of Patent Document 1, the handwritten images of other users are displayed in a color different from that of their own handwritten images, but the handwritten images overlap and become difficult to see because, for example, a plurality of users simultaneously input in the same region. That is not expected.
Therefore, an object of the present invention is to prevent handwritten images of the users from overlapping even if the users simultaneously input the handwritten images to display them on the shared screen.

In order to achieve the above object, an information acquisition unit that acquires information of user's handwriting input from a plurality of user devices and a handwritten image indicated by each of the information acquired from a plurality of the user devices are displayed on a shared screen. A generation unit that generates screen data, and arranges a first handwritten image indicated by the information acquired from the first user device in a first area of the shared screen where handwriting input by the first user device is permitted. However, the second handwritten image indicated by the information acquired from the second user device during the period in which the information is acquired from the first user device does not overlap the first area on the shared screen and (2) An information processing apparatus is provided that includes a generation unit that generates the screen data by arranging it in a second area where handwriting input by the user device is permitted, and an output unit that outputs the generated screen data.

Further, the generation unit may set an area having a size or shape corresponding to the information of the handwriting input acquired from the first user device as the first area.
Furthermore, a recognition unit that recognizes the character when the first handwritten image is a character is provided, and the generation unit determines the size of the character or the character when the first handwritten image is recognized as the character. The line-up direction may be used as the information of the handwriting input.

Further, the image processing apparatus further includes a storage unit that stores a history of the acquired information of the handwriting input, and the generation unit responds to a handwritten image estimated to be input from the history stored for the first user device. A region of size or shape may be the first region.
Furthermore, when the first handwritten image does not fit in the first area, the generation unit enlarges the first area to a size that the first handwritten image fits, and enlarges the second area. You may move to the position which does not overlap with 1 area|region.

Further, when the input of at least one of the first handwritten image and the second handwritten image is completed and there is a gap between both images, the generation unit is configured to fill the gap with the first handwritten image. One or both of the image and the second handwritten image may be moved.
Furthermore, an image acquisition unit that acquires a designated image designated by the user from the first user device is provided, and the generation unit generates the screen data in which the obtained designated image is arranged in the first area. You may.

According to the present invention, even if each user inputs handwritten images at the same time in order to display them on the shared screen, the handwritten images of those users can be prevented from overlapping.

The figure showing the whole structure of the handwritten image sharing system which concerns on an Example. Diagram showing the hardware configuration of the screen sharing server device Diagram showing the hardware configuration of the user device Diagram showing the functional configuration realized by the handwritten image sharing system The figure showing an example of the specified method of the pointing position. The figure showing an example of the determination method of an input permission area The figure showing an example of the handwritten image displayed on another user apparatus. The figure showing another example of the determination method of an input permission area. The figure showing an example of the operation procedure of each apparatus in a shared display process. The figure showing an example of the enlarged input permission area. The figure showing an example of the reduced handwritten image. The figure showing an example of the handwritten image moved so as to reduce the gap. The figure showing the functional structure which the handwritten image sharing system of a modification implement|achieves. Figure showing an example of accumulated history The figure showing the functional structure which the handwritten image sharing system of a modification implement|achieves. Figure showing an example of the displayed designated image The figure showing an example of the whole structure of the handwriting image sharing system of a modification. Diagram showing the functional configuration realized by the handwritten image sharing system

[1] Example FIG. 1 shows an overall configuration of a handwritten image sharing system 1 according to an example. The handwritten image sharing system 1 displays a shared screen on a user device used by each of a plurality of users, and when the user performs handwriting input on the shared screen, not only the shared screen of the own user device but also the user devices of other users. In this system, a handwritten image input by handwriting is also displayed on the shared screen of (1) to share the handwritten image input by each user with a plurality of users.

The handwritten image sharing system 1 includes a network 2, a screen sharing server device 10, and user devices 20-1, 20-2, and 20-3 (hereinafter referred to as “user device 20” unless otherwise specified). The network 2 is a communication system including a mobile communication network, the Internet, and the like, and mediates exchange of data between devices that access the own system. The screen sharing server device 10 is accessing the network 2 by wire communication, and the user device 20 is accessing by wireless communication. The access to the network 2 may be wired communication or wireless communication.

The screen sharing server device 10 is an information processing device that performs processing for sharing a screen for handwriting input displayed by each user device 20. The user device 20 displays a screen for handwriting input shared by the screen sharing server device 10, that is, a shared screen of handwritten images, accepts handwriting input by the user, and draws a handwritten image drawn by the handwriting input. It is an information processing device that displays a handwritten image of a user who uses it and a handwritten image of a user who uses another user device 20.

FIG. 2 shows the hardware configuration of the screen sharing server device 10. The screen sharing server device 10 is a computer including a processor 11, a memory 12, a storage 13, a communication device 14, an input device 15, an output device 16, and a bus 17. Note that the word "apparatus" here can be read as a circuit, a device, a unit, or the like. In addition, each device may include one or more devices, or may not include some devices.

The processor 11 operates, for example, an operating system to control the entire computer. The processor 11 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, a calculation device, a register, and the like. Further, the processor 11 reads a program (program code), a software module, data, and the like from the storage 13 and/or the communication device 14 into the memory 12, and executes various processes according to these. The number of processors 11 that execute various processes may be one, or two or more, and the two or more processors 11 may execute various processes simultaneously or sequentially. Further, the processor 11 may be implemented by one or more chips. The program may be transmitted from the network via a telecommunication line.

The memory 12 is a computer-readable recording medium, and includes at least one of a ROM (Read Only Memory), an EPROM (Erasable Programmable ROM), an EEPROM (Electrically Erasable Programmable ROM), a RAM (Random Access Memory), and the like. May be done. The memory 12 may be called a register, a cache, a main memory (main storage device), or the like. The memory 12 can store the above-mentioned program (program code), software module, data, and the like.

The storage 13 is a computer-readable recording medium, and is, for example, an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (for example, a compact disk, a digital versatile disk, a Blu-disk). ray(R) disk), smart card, flash memory (e.g., card, stick, key drive), floppy(R) disk, magnetic strip, and the like. The storage 13 may also be called an auxiliary storage device. The above-described storage medium may be, for example, a database including the memory 12 and/or the storage 13, a server, or another appropriate medium. The storage 13 stores a program of an application (handwriting application) that realizes handwriting input.

The communication device 14 is hardware (transmission/reception device) for performing communication between computers via a wired and/or wireless network, and is also called, for example, a network device, a network controller, a network card, a communication module, or the like. The input device 15 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives an input from the outside.

The output device 16 is an output device (for example, a display, a speaker, an LED lamp, etc.) that performs output to the outside. The input device 15 and the output device 16 may be integrated (for example, a touch screen). Further, the respective devices such as the processor 11 and the memory 12 are accessible to each other via a bus 17 for communicating information. The bus 17 may be configured with a single bus or different buses between devices.

The screen sharing server device 10 is a hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array). It may be configured to include hardware, and a part or all of each functional block may be realized by the hardware. For example, the processor 11 may be implemented with at least one of these hardware.

FIG. 3 shows the hardware configuration of the user device 20. The user device 20 is a computer including a processor 21, a memory 22, a storage 23, a communication device 24, an input/output device 25, and a bus 26. The processor 21 to the communication device 24 and the bus 26 are hardware common to the devices of the same name shown in FIG. The input/output device 25 includes an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a touch sensor, and the like) that receives an input from the outside, and an output device that performs an output to the outside (for example, a display, a speaker, and an LED). Lamp, etc.). In the present embodiment, the user device 20 includes a touch screen which is an input/output device in which an input device and an output device are integrated.

The functions described below are realized by the processor 11 of the screen sharing server device 10 and the processor 21 of each user device 20 executing a program to control each unit of each device.
FIG. 4 shows a functional configuration realized by the handwritten image sharing system 1. The screen sharing server device 10 includes a handwriting input information acquisition unit 101, a character recognition unit 102, a region determination unit 103, a screen data generation unit 104, and a screen data output unit 105. The user device 20 includes a handwriting input operation reception unit 201, a handwriting input information output unit 202, and a handwriting input screen display unit 203. Although only one user device 20 is shown in FIG. 4, it is assumed that all user devices 20 have the same configuration.

The handwriting input operation receiving unit 201 of the user device 20 receives an operation of handwriting input performed by the user by instructing the position on the display surface. Upon receiving the handwriting input operation, the handwriting input operation receiving unit 201 specifies the position (instructed position) on the display surface instructed by the operation, for example, by the following method.

FIG. 5 shows an example of a method of specifying the designated position. FIG. 5A shows a state in which the user's finger B1 indicates the position on the display surface A1 displaying the handwriting input screen. For example, as shown in FIG. 5B, the handwriting input operation receiving unit 201 identifies the contact area F1 of the finger B1 that is in contact with the display surface A1, and identifies the center of the contact area F1 as the designated position G1. To do. Note that the method for identifying the designated position is not limited to this, and other known techniques may be used.

The handwriting input operation reception unit 201 supplies position information indicating the specified pointing position to the handwriting input information output unit 202. As shown in the figure, since the display surface A1 is a rectangle, the handwriting input operation receiving unit 201 has a certain corner of the display surface A1 (in this embodiment, the upper left corner of the displayed character is the upper left corner). Is the origin P0, and the coordinates of the designated position G1 in the coordinate system in which one side forming the angle is the X axis and another side is the Y axis are supplied as position information.

In this coordinate system, the direction of the arrow indicating the X axis (rightward direction) in the figure is the positive direction of the X axis, and the direction of the arrow indicating the Y axis (downward direction) is the positive direction of the Y axis. .. The handwriting input operation reception unit 201 repeatedly specifies the designated position and supplies the position information, for example, at regular time intervals. It is desirable that the time intervals used here be narrow enough so that an image (handwritten image such as a character or a picture) represented by the locus of the designated position can be recognized.

The handwriting input information output unit 202 outputs the information of the handwriting input received by the handwriting input operation receiving unit 201 to the screen sharing server apparatus 10. The handwriting input information output unit 202 handwrites position information repeatedly supplied from the handwriting input operation reception unit 201, that is, position information indicating a plurality of designated positions specified by the handwriting input operation reception unit 201, each time the position information is handwritten. Output as input information. The handwritten input information is information representing a handwritten image (characters or pictures, etc.) input by handwriting, and the loci of a plurality of positions indicated by the plurality of position information sequentially output by the handwriting input information output unit 202 are: Represents a handwritten image.

The handwriting input information acquisition unit 101 of the screen sharing server device 10 acquires the handwriting input information of the user from a plurality of user devices. The handwritten input information acquisition unit 101 is an example of the “information acquisition unit” in the present invention. In the present embodiment, the handwriting input information acquisition unit 101 uses the above-mentioned position information output by the user devices 20-1, 20-2 and 20-3 shown in FIG. 1 as the handwriting input of the user who uses each device. Get as information. The handwritten input information acquisition unit 101 supplies the acquired information to the character recognition unit 102 and the area determination unit 103 every time the handwritten input information is acquired.

The character recognition unit 102 recognizes a character when the handwritten image indicated by the handwriting input information acquired by the handwriting input information acquisition unit 101 represents the character. The character recognition unit 102 is an example of the “recognition unit” in the present invention. The character recognition unit 102 uses, for example, the technique of OCR (Optical character recognition) based on the handwriting input information supplied from the handwriting input information acquisition unit 101, that is, the position information indicating the pointing position instructed by the user. The character indicated by the locus of the designated position is recognized.

When recognizing a character, the character recognition unit 102 supplies information indicating that the character has been recognized and the supplied handwriting input information to the area determination unit 103. When the character recognition unit 102 does not recognize the character, the character recognition unit 102 supplies information notifying that effect and the supplied handwriting input information to the area determination unit 103. In this embodiment, a case where a character is recognized will be described.

The area determination unit 103 determines the shape and size of the area (input permitted area) where handwriting input by each user device 20 is permitted on the shared screen, for each user device 20. A method of determining the input permission area will be described with reference to FIGS. 6 to 8.
FIG. 6 shows an example of a method of determining the input permission area. In FIG. 6 (a), the user 1 to use the user device 20-1, display the handwriting input screen on the display surface A1, was entered handwritten image B11 that "Hello!" Is shown. The handwritten image B11 is displayed by the operations of the screen data generation unit 104, the screen data output unit 105, and the handwriting input screen display unit 203, which will be described later, and these operations will be described later.

When the information indicating that the character has been recognized is supplied, the area determination unit 103 supplies a handwritten image indicated by the supplied handwriting input information, that is, a quadrangle C1 circumscribing the handwritten character (each side is shown in FIG. 5). A side length L11 along the X axis and a side length L12 along the Y axis of a quadrangle along the X axis and the Y axis of the coordinate system centered on the origin P0 are shown in FIG. 6B. Calculate to. The area determination unit 103 determines that the handwritten characters are drawn horizontally when L11>L12, and the handwriting characters are drawn vertically when L11<L12.

In this example, the area determination unit 103 determines that the handwritten character is written horizontally. In this case, the area determination unit 103 determines a horizontally long rectangle as the input permission area. Specifically, the area determination unit 103 extends the upper side of the quadrangle C1 to the left and right to the edge of the display surface A1, and makes the size in the vertical direction larger than the quadrangle C1 by a predetermined ratio (in this example, N(N Is a natural number and multiplied by 4) is determined as the input permission area D1. In the present embodiment, the horizontally written character is written to the right end and the next character is written on the lower side when a line feed is performed, and the area determination unit 103 causes the area determination unit 103 to input the area so that the area extends below the handwritten character that has already been input. To decide.

When the handwriting character is determined to be vertically written, the region determination unit 103 determines a vertically long rectangle as the input permission region. In that case, the area determining unit 103 extends the right side of the quadrangle circumscribing the handwritten character up and down to the end of the display surface A1, and makes the horizontal size larger than the quadrangle by a predetermined ratio. To decide. In this embodiment, vertically written characters are written to the lower end, and when a line feed is performed, the next character is written on the left side. The area determining unit 103 sets the input permission area so that the area extends to the left side of the input handwritten character. decide.

Since the user 1 is permitted handwriting input in the input permission area D1 determined in the example of FIG. 6, the handwritten image handwritten by the user 1 in the input permission area D1 is the handwritten image B12 shown in FIG. 6D. Is displayed. On the other hand, other users are not permitted to input by handwriting in the input permission area D1. Therefore, for example, hand-drawn image user 2 to use the user device 20-2 has handwriting input is displayed in the region outside the input permission area D1 as handwritten image B21 that "Hello!" Depicted in Fig. 6 (e) Will be.

In FIG. 6E, an in-writing image H1 indicating that another user is writing is displayed at the lower right of the handwritten image B21. This in-writing image H1 represents the range of the input permission area permitted by the user 2. In this example, an image of a handwritten character string prepared in advance is used as the in-writing image H1 instead of the image handwritten by the user 2.

FIG. 7 shows an example of a handwritten image displayed on another user device. In FIG. 7 (a), the display surface A2 of the user device 20-2, the hand-drawn image B11 is displayed that is handwriting input at the user device 20-1 "Hello!". Further, the in-writing image H2 is displayed on the display surface A2. The in-writing image H2 is displayed at the lower end of the input permission area D1 described in the description of FIG. 6, and represents the lower end of the input permission area D1. Since the handwritten image is not displayed even if input is made in the area sandwiched between the handwritten image B11 and the in-writing image H2, the user 2 starts handwriting input from the lower side of the in-writing image H2.

In FIG. 7B, the handwritten image B21 input by the user 2 is displayed on the display surface A2 in an area outside the input permission area D1. Since the handwriting image B21 is recognized as a character by the character recognizing unit 102, the region determining unit 103 causes the region determining unit 103 to determine the side length L21 of the quadrangle C2 circumscribing the handwriting image B21 along the X-axis and The length L22 of the side along the Y axis is calculated as shown in FIG.

Since L21>L22, the area determination unit 103 determines that the character represented by the handwritten image B21 is in horizontal writing, and based on the rectangle C2 as in the input allowed area D1, the input allowed area D2 (the vertical size is N times as large as L22). Square) is determined as shown in FIG. At this time, the in-writing image H1 displayed on the display surface A1 in FIG. 6E is not displayed on the display surface A2.

Further, the area determination unit 103 determines the input permission area based on the handwriting input information supplied from the handwriting input information acquisition unit 101 before the character is recognized by the character recognition unit 102.
FIG. 8 shows another example of the method of determining the input permission area. FIG. 8A shows that the user 2 has designated the designated position G2 on the display surface A2 of the user device 20-2 and started the handwriting input. In the area determination unit 103, for example, the designated position G2 indicated by the information of the handwriting input is located at the center in the vertical direction, and the size in the vertical direction is predetermined (determined as the size L1 in this example). A quadrangle having the horizontal width of the display surface as the horizontal size is determined as the input permission area D3.

At this time, on the display surface A1 of the user device 20-1, only the in-writing image H3 is displayed in the input permission area D3 as shown in FIG. 8B, and the handwritten image input by the user 2 is not yet displayed. .. In this case as well, the in-writing image H3 represents the range of the input permission area permitted by the user 2. The area determination unit 103 supplies the area information indicating the input permission area determined as described above to the screen data generation unit 104 together with the supplied handwriting input information.

The screen data generation unit 104 generates screen data in which a handwritten image indicated by the supplied handwriting input information, that is, the handwritten input information acquired from each of the plurality of user devices 20 is displayed on the shared screen. Further, the screen data generation unit 104, for example, the user 1 of the user device 20-1 (an example of the “first user device” of the present invention) performs handwriting input, and the user device 20 is in a period during which the handwriting input is performed. -2 (an example of the "second user device" of the present invention), when user 2 performs handwriting input, screen data is generated as follows.

The screen data generation unit 104 arranges the handwritten image indicated by the handwriting input information acquired from the user device 20-1 in the first input permission area on the shared screen where the handwriting input by the user device 20-1 is permitted. In the following, the handwritten image previously input in this manner is referred to as a “first handwritten image”, and the input permission area permitted by the user device 20 in which the first handwritten image is input is referred to as a “first input permission area”. .. In the example of FIG. 6, the screen data generation unit 104 arranges the handwritten image B11, which is the first handwritten image, with the input permission area D1 determined by the area determination unit 103 as the first input permission area.

In addition, the screen data generation unit 104 displays the handwritten image indicated by the handwritten input information acquired from the user device 20-2 during the period in which the handwritten input information is acquired from the user device 20-1 as a first image on the shared screen. The screen data is generated by arranging it in an input permission area that does not overlap the input permission area and in which handwriting input by the user device 20-2 is permitted. Hereinafter, the handwritten image thus input later is referred to as a “second handwritten image”, and the input permission area permitted by the user device 20 to which the second handwritten image is input is referred to as a “second input permission area”. ..

The “period during which the handwritten input information is acquired” here means only the moment when the handwritten input information acquisition unit 101 acquires the handwritten input information (the moment when the packet of data indicating the handwritten input information is received). Rather, it means the entire period in which the user 1 continuously inputs the handwriting to the user device 20-1 and the information of the handwriting input is intermittently acquired. The screen data generation unit 104 determines that the period in which the handwritten input information is acquired has ended when, for example, the period in which the handwritten input information is not acquired exceeds the threshold value.

In the example of FIG. 6 (d), the has been instructed to until the user 1 is completed the input of the handwritten image B12 of "Hello!", "It's good weather today!" From the start of the input of the handwritten image B11 that The position information indicating the position is intermittently acquired as handwritten input information. In the example of FIG. 7, the handwriting input information (position information indicating the locus of the handwritten image B12 of “Today is good weather!”) is acquired from the user device 20-1 from the user device 20-2. of handwriting input information ( "Hello!" position information indicating the trajectory of the handwritten image B21 that) has been acquired.

In this case, the screen data generation unit 104 uses the input permission area D2, which is determined as an area that does not overlap the input permission area D1 that is the first input permission area, as the second input permission area and is the second handwritten image B21. Are arranged. In the present embodiment, since the area determination unit 103 determines the input permission area as described in the description of FIG. 6B, the screen data generation unit 104 causes the screen data generation unit 104 to obtain the handwriting input information acquired from the user device 20-1. An area having a size and shape corresponding to the input permission area (the input permission area D1 in the example of FIG. 6B) is set.

More specifically, when the handwriting image indicated by the handwriting input information acquired from the user device 20-1 is recognized as a character by the character recognition unit 102, the screen data generation unit 104 recognizes the character (see FIG. In the example of (), the area having a size and shape corresponding to the size of the handwritten image B11) and the direction in which the characters are arranged is set as the input permission area (the input permission area D1 in the example of FIG. 6B).

The input permission area D1 is an area in which the vertical size is larger than the vertical size of the handwritten image B11 by a predetermined ratio (N times in the example of FIG. 6). The input permission area D1 is a horizontally long rectangular area because the character represented by the handwritten image B11 is horizontally written. Further, when the handwritten character is written vertically, the input permitted area has a vertically long rectangular shape as described above, and the horizontal size is an area in which the horizontal size of the handwritten character is increased by a predetermined ratio. Become. The screen data generation unit 104 supplies the screen data generated as described above to the screen data output unit 105.

The screen data output unit 105 outputs the screen data generated by the screen data generation unit 104. The screen data output unit 105 is an example of the “output unit” in the present invention. In the handwritten image sharing system 1, a group of user devices 20 that supply handwritten images is registered. The screen sharing server device 10 stores the destinations (IP (Internet Protocol) addresses and the like) of the user devices 20 in the same group in association with information (for example, group ID (Identification)) for identifying the group.

In this embodiment, the user devices 20-1, 20-2, 20-3 are registered as the same group. When the handwriting input information acquisition unit 101 acquires the handwriting input information from the user devices 20-1 and 20-2 as in the examples of FIGS. 6 and 7, the screen data output unit 105 outputs the screen data generation unit 104. The supplied screen data is output to the user devices 20-1, 20-2, 20-3.

The handwriting input screen display unit 203 of the user device 20 displays the screen indicated by the screen data output from the screen sharing server device 10. The handwriting input screen display unit 203 of the user device 20-1 displays a screen as shown in FIG. 6, for example, and the handwriting input screen display unit 203 of the user device 20-2 displays a screen as shown in FIG. 7, for example. To do. The handwriting input screen display unit 203 of the user device 20-3 also displays the handwritten image input by handwriting by the user 1 and the user 2. Moreover, when the user 3 who uses the user device 20-3 performs handwriting input, the handwriting input screen display unit 203 of each user device 20 displays the handwritten image.

The screen sharing server device 10 and each user device 20 perform sharing display processing for displaying a user's handwritten image on the shared screen based on the above configuration.
FIG. 9 shows an example of an operation procedure of each device in the shared display processing. Although only the user devices 20-1 and 20-2 are shown in FIG. 9, the user device 20-3 also operates similarly to these. This operation procedure is started, for example, when each user operates his or her own user device 20 to activate the handwriting application and perform an operation of drawing a handwritten image.

First, the user device 20 (handwriting input operation receiving unit 201) receives an operation of handwriting input performed by the user instructing the display surface (steps S11 and S21). The common operations of the user devices 20-1 and 20-2 are collectively represented as described above. Next, the user device 20 (handwriting input information output unit 202) outputs the information of the handwriting input for which the operation is accepted to the screen sharing server device 10 (steps S12 and S22).

The screen sharing server device 10 (handwriting input information acquisition unit 101) acquires the handwriting input information output from each of the user devices 20-1 and 20-2 (steps S31 and S32). Next, the screen sharing server apparatus 10 (character recognition unit 102) recognizes the character if the handwritten image indicated by the acquired handwriting input information is a character (step S33). Subsequently, the screen sharing server apparatus 10 (area determining unit 103) determines an area having a size and shape corresponding to the size of the recognized character and the direction in which the characters are arranged as the input permission area (step S34).

Next, the screen sharing server device 10 (screen data generation unit 104) generates screen data indicating a shared screen in which a handwritten image is arranged in the determined input permission area (step S35). Subsequently, the screen sharing server device 10 (screen data output unit 105) outputs the generated screen data to the plurality of user devices 20 in the same group (steps S36 and S37). The user device 20 (handwriting input screen display unit 203) displays the shared screen indicated by the output screen data (steps S13 and S23). Then, the user device 20 returns to steps S11 and S21 and continues the operation. In this way, the handwritten image input by the user by handwriting is displayed with the handwriting input screen of each user device 20 as a shared screen.

In this embodiment, even if each user simultaneously inputs a handwritten image to display it on the shared screen, the handwritten image B11 of the user 1 is displayed in the input permission area D1 as shown in the example of FIG. Since the handwritten image B21 of No. 2 is displayed in the input permission area D2 that is determined as an area that does not overlap the input permission area D1, it is possible to prevent the handwritten images of the users 1 and 2 from overlapping.

Further, in this embodiment, when the user inputs a handwritten character, an area having a size corresponding to the size of the character becomes the input permission area. As a result, for example, a large input permission area is secured for a user who writes large characters, and a small input permission area is secured for a user who writes small characters. The difference in the number of characters that can be input in the input permission area between users can be reduced.

Further, in this embodiment, when the user inputs a handwritten character, an area having a shape corresponding to the direction in which the character is arranged becomes the input permission area. Thus, for example, a horizontally long rectangular input permission area is secured for a user who writes characters horizontally, and a vertically long input permission area is secured for a user who writes characters vertically. Compared with the case where the shape is constant, it is possible to prevent the difficulty of writing because a line break is required immediately after writing a character.

[2] Modified Example The above-described embodiment is merely an example of the implementation of the present invention, and may be modified as follows. Further, the embodiment and each modification may be implemented in combination as necessary.

[2-1] Expansion of Input Allowed Area The handwritten image of the user may not fit in the input allowed area secured as described in the embodiment. In that case, the input permission area may be enlarged to accommodate the handwritten image. In this modification, when the handwritten image does not fit in the input permission area, the area determination unit 103 determines the input permission area enlarged to a size in which the handwritten image fits.

Therefore, when the handwritten image does not fit in the input permission area, the screen data generation unit 104 also generates screen data indicating a screen in which the input permission area is enlarged to a size in which the handwritten image fits.
FIG. 10 shows an example of the enlarged input permission area. Figure 10 (a), the are displayed hand-drawn image B13 that the user 1 has handwriting input "Hello! It's good weather today!" To the input enable area D4 of the display surface A1 of the user device 20-1, input and in the image H4 writing the handwritten image B24 that the user 2 has handwriting input "Hello!" is on the lower side of the permit area D4 are displayed in the input permission area D5.

In FIG. 10B, a handwritten image B14, which is “DO”, which is continuously input by the user 1 below the handwritten image B13, is displayed on the display surface A1. Since this handwritten image B14 is not stored in the input permission area D4, the screen data generation unit 104 enlarges the input permission area D4 to a size in which the handwritten image B14 can be stored as an input permission area D6, and handwrites in the input permission area D6. The screen data in which the image B14 is arranged is generated.

Further, the screen data generation unit 104 expands the input permission area to D6, so that the input permission area D5 in which the handwritten image B24 displayed in the enlarged portion is arranged does not overlap the input permission area D6. Have moved to. In this way, the screen data generation unit 104 stops storing the first handwritten image (the previously input handwritten image. Handwritten image B14 in this example) in the first input permission area (input permission area D4 in this example). In this case, the first input permission area is enlarged to fit the first handwritten image, and the second input permission area (handwritten image B24 input later. In this example, handwritten image B24) is arranged in the second input permission area ( In this example, the input permission area D5) is moved to a position that does not overlap the enlarged first input permission area.

In this modification, the input permission area is expanded to D6 in this way, and the input permission area D5 is moved (the handwritten image B24 and the writing image H4 arranged in the input permission area D5 are also moved). As shown in FIG. 10C, the handwritten image B15 “Let's go play somewhere!” can be drawn without overlapping the handwritten image B24 of the user 2.

The input permission area determined before the characters are recognized as described in the example of FIG. 8 is expanded to a larger input permission area when the characters are recognized. If this input permission area is set as the first input permission area, in this case also, the screen data generation unit 104 does not overlap the second input permission area in which the second handwritten image is arranged with the enlarged first input permission area. Just move it to the position. As a result, even if the second handwritten image that the user 2 started to draw is displayed before the first handwritten character of the user 1 is recognized, the user 1 does not overlap the second handwritten character with the first handwritten character. I can continue to write.

[2-2] Reduction of handwritten image When the handwritten image of the user does not fit in the input permission area as described above, the handwritten image may be reduced instead of enlarging the input permission area. In this modified example, when the handwritten image does not fit in the input permission area, the screen data generation unit 104 generates screen data indicating a screen in which the handwritten image is reduced in size and placed in the input permission area.

FIG. 11 shows an example of a reduced handwritten image. In the example of FIG. 11, the screen data generation unit 104 generates screen data showing a screen in which the handwritten images B13 and B15 reduced to fit in the input permission area D4 shown in FIG. 10 are arranged in the input permission area D4. .. In the present modification, when the handwritten image does not fit in the input permission area, it is possible to continue drawing the handwritten image without overlapping the handwritten image of the other user without moving the handwritten image of the other user.

[2-3] Reduction of Gap If there are few handwritten images input to the secured input permission area, the input permission area that has not been input remains as a gap with the handwritten image of another user. An operation of reducing this gap may be performed.

In the present modification, when the screen data generation unit 104 has completed the input of at least one of the first handwritten image and the second handwritten image and a gap has been created in both images, the screen data generation unit 104 first sets the position to fill the gap. Screen data indicating a screen obtained by moving one or both of the first handwritten image and the second handwritten image is generated.

FIG. 12 shows an example of a handwritten image moved so as to reduce the gap. Figure 12 (a), the display surface A2 of the user device 20-2, represents the handwritten image B15 that the user 1 has handwriting input "Hello!", The range of the input enable area D7 where the handwritten image B15 is arranged The in-writing image H5 is displayed. In addition, the hand-drawn image B17 of "Hello!" User 2 handwriting input to the input enabled area D8, which is secured on the lower side of the input enable area D7 is displayed.

In this example, the user 2 continues to input the handwriting even after the state of FIG. 12A, but the user 1 has stopped the handwriting input, so that the in-writing image H5 is displayed as shown in FIG. 12B. It is no longer being done. Therefore, between the handwritten image B15 and the handwritten image B17, a gap E1 having a length L31 in the vertical direction is formed as shown in FIG. In this case, the screen data generation unit 104 has completed the input of the handwritten image B15 and created a gap between the handwritten images B15 and B17. Therefore, a screen showing the screen in which the handwritten image B15 is moved to a position to fill the gap. Generate data.

The screen represented by the screen data thus generated is displayed on the display surface A2 as shown in FIG. In this example, the handwritten images B15 and B17 are displayed separated by a gap E2 (the length in the vertical direction is L32 (L32<L31)). This makes it possible to reduce the gap between the handwritten images displayed on the shared screen and reduce the number of handwritten images displayed on the shared screen, as compared to the case where the gap is not reduced. You can

[2-4] Method of Determining Input Allowed Area The area determining unit 103 may determine the input allowed area by a method different from the above example.
FIG. 13 shows a functional configuration realized by the handwritten image sharing system 1a of this modification. The handwritten image sharing system 1a includes a screen sharing server device 10a and a user device 20. The screen sharing server device 10a includes an information storage unit 106 and a handwritten image estimation unit 107 in addition to the units illustrated in FIG.

In this modification, when the character recognition unit 102 recognizes a character, the character storage unit 106 supplies information indicating that the character has been recognized and the supplied handwriting input information to the information storage unit 106, and may not recognize the character. , And supplies the supplied handwritten input information to the information storage unit 106. The information storage unit 106 stores the supplied handwriting input information, that is, the history of the handwriting input information acquired by the handwriting input information acquisition unit 101.

FIG. 14 shows an example of the accumulated history. In the example of FIG. 14, the information storage unit 106 inputs the type of the handwritten image, the size of the handwritten image, and the input for each user device (user devices 20-1 and 20-2 in this example) that uses the user device 20. The date and time are stored in association with each other. In the present modification, when the handwriting input information output unit 202 of the user device 20 outputs handwriting input information, for example, a device ID is also output as identification information for identifying the own device, and the device ID is the handwriting input information acquisition unit. The information is supplied to the information storage unit 106 via the 101 and the character recognition unit 102 (the device IDs of the user devices 20-1 and 20-2 are “U001” and “U002”, respectively).

Like the screen data generation unit 104, the information storage unit 106 determines the period during which the handwritten input information is acquired, and determines the type and size of the handwritten image based on the handwritten input information supplied during that period. .. If the character is recognized by the character recognition unit 102, the information storage unit 106 determines whether horizontal writing is vertical writing as in the area determination unit 103 described in the description of FIG. For example, if "horizontal writing" is determined to be vertical writing, "vertical writing" is stored as the type of handwritten image. Further, the information storage unit 106 stores “picture” as the type of the handwritten image if the character is not recognized by the character recognition unit 102.

The information storage unit 106 also stores the size of a quadrangle circumscribing the handwritten image indicated by the handwritten input information as the size of the handwritten image. In addition, the information storage unit 106 stores, for example, the date and time when the period in which the information of the handwritten input is acquired starts or ends as the input date and time. The information storage unit 106 stores these pieces of information in association with the device ID supplied together with the handwritten input information.

The handwritten image estimation unit 107 estimates a handwritten image estimated to be input by the user, from the history of handwritten input accumulated for the user device 20 used by the user. The handwritten image estimation unit 107 estimates that the type of the handwritten image that is most stored in association with the device ID is input by the user who uses the user device 20 corresponding to the device ID, for example. Presumed to be an image.

For example, for the user device 20-1 with the device ID “U001”, the handwritten image estimation unit 107 estimates that “horizontal characters” are drawn because all types of handwritten images are “horizontal characters”. Further, the handwritten image estimation unit 107 estimates that the “drawing” is drawn for the user device 20-2 having the device ID “U002”, because the “vertical writing” is once and the “drawing” is twice. To do. The handwritten image estimation unit 107 supplies information indicating the handwritten image thus estimated (type of handwritten information) to the area determination unit 103.

The area determination unit 103 sets an area having a size and shape corresponding to the supplied type of handwritten image, that is, a handwritten image estimated to be input from the history of handwriting input accumulated for the user device 20, as an input permission area. To decide. For example, when “horizontal writing” is estimated, the area determination unit 103 determines a rectangle having the full width in the horizontal direction of the display surface as the shape of the input permission area. Then, the area determination unit 103 calculates an average value of the vertical size of the handwritten image stored in association with the “horizontal characters”, and determines the average value as the vertical size.

In the example of FIG. 14, the area determination unit 103 determines (L41+L43+L45)/3 based on the vertical size of the handwritten image drawn as horizontally-written characters for the user device 20-1 having the device ID “U001”. A rectangle whose size is in the vertical direction is determined as the input permission area. Further, for example, when “vertical writing characters” are estimated, the area determining unit 103 determines a rectangle having the full width in the vertical direction of the display surface as the shape of the input permission area, and handwriting drawn as vertical writing characters. A rectangle whose horizontal size is the average value of the horizontal size of the image is determined as the input permission area.

In addition, when the “picture” is estimated, the area determination unit 103 calculates the average value of the vertical size and the average value of the horizontal size of the handwritten image drawn as a picture, and calculates the average values. A rectangle having a vertical size and a horizontal size is determined as the input permission area. In the example of FIG. 14, the area determination unit 103 vertically sets the average value (L53+L55)/2 of the vertical size of the handwritten image drawn as a picture for the user device 20-2 having the device ID “U002”. A rectangle having the horizontal size and the average horizontal size (L54+L56)/2 is determined as the input permission area.

Since the area determination unit 103 determines the input permission area as described above, the screen data generation unit 104 determines the size and the size according to the handwritten image estimated to be input from the history of handwriting input accumulated for the user device 20. The shape area will be the input permission area. As a result, the tendency of the user's handwriting input even after the handwriting input information acquisition unit 101 starts acquisition of handwriting input information and before the character recognition unit 102 recognizes a character, as in the example of FIG. It is possible to secure an area that matches the above as an input permission area.

In the above example, the average value of the sizes is calculated, but the present invention is not limited to this, and the maximum value, the minimum value, or the intermediate value of the sizes may be calculated. Further, the history used for estimation may be limited to only the history accumulated in a specific period (for example, the past one year). In that case, it is possible to secure an area suitable for the recent tendency of the user's handwriting input as the input permission area.

[2-5] Display of images other than handwritten images Images other than handwritten images (images drawn by the operation accepted by the handwriting input operation acceptance unit 201 of the user device) may be displayed in the input permission area.
FIG. 15 shows a functional configuration realized by the handwritten image sharing system 1b of this modification. The handwritten image sharing system 1b includes a screen sharing server device 10b and a user device 20b. The screen sharing server device 10b includes a designated image acquisition unit 108 in addition to the units illustrated in FIG. The user device 20b includes a designated image output unit 204 in addition to the units shown in FIG.

The designated image output unit 204 designates an image displayed on the display surface, or displays a file management application or the like to designate an image file, and pastes the designated image in the input permission area (for example, during display). (Operations of dragging the image or image file to the input permission area and the like) are received, the image designated by these operations is output to the screen sharing server device 10b. The designated image acquisition unit 108 acquires, from the user device 20, the image thus output, that is, the designated image designated by the user. The designated image acquisition unit 108 is an example of the “image acquisition unit” in the present invention.

The designated image acquisition unit 108 supplies the acquired designated image to the screen data generation unit 104. The screen data generation unit 104 generates screen data in which the designated image acquired by the designated image acquisition unit 108 is arranged in the input permission area.
FIG. 16 shows an example of the designated image displayed. In the example of FIG. 16, the screen data generation unit 104 generates screen data in which the designated image J1 is arranged in the input permission area D1 shown in FIG. 6C. The designated image output unit 204 outputs the position information indicating the pasting position where the designated image is pasted together with the designated image, so that the screen data generation unit 104 displays the screen in which the designated image is arranged at the pasted position of the input permission area. Data may be generated.

In addition, when the designated image extends beyond the input permission area, the screen data generation unit 104 may generate screen data in which the designated image is reduced to a size that fits in the input permission area, or a size that fits the designated image. You may generate the screen data which expanded the input permission area. Further, as the designated image, a character represented by a font input with a keyboard or the like may be used. According to this modification, it is possible to display a designated image such as a photograph, CG (Computer Graphics) and characters represented by a font together with a handwritten image on the shared screen of the handwritten image.

[2-6] Device that realizes each unit The device that realizes each unit illustrated in FIG. 4 is not limited to the above-described device. For example, one or both of the functions of the character recognition unit 102 and the area determination unit 103 may be realized by another server device. The user device may also realize the function realized by the screen sharing server device.

FIG. 17 shows an example of the overall configuration of the handwritten image sharing system 1c of this modification. The handwritten image sharing system 1c includes a network 2 and user devices 20c-1, 20c-2, 20c-3 (hereinafter referred to as "user device 20c" unless otherwise specified). Each user device 20c communicates with each other via the network 2. Note that the user devices 20c communicate with each other by P2P (Peer to Peer), or one user device 20c functions as a wireless router so that other user devices 20c communicate with each other via the user device 20c. May be.

FIG. 18 shows a functional configuration realized by the handwritten image sharing system 1c. The user device 20c includes not only the units included in the user device 20 in FIG. 4 but also each unit included in the screen sharing server device. The handwriting input information acquisition unit 101 of the user device 20c acquires handwriting input information from the own device and another user device 20c, and the screen data generation unit 104 uses screen data based on the acquired handwriting input information. To generate. The screen data output unit 105 of the user device 20c displays the shared screen by outputting the screen data to the handwriting input screen display unit 203 of the own device. According to this example, even if the server device is not provided, it is possible to display the handwritten image on the shared screen of each user device and share the handwritten image with a plurality of users.

[2-7] Input Allowed Area The input allowed area is not limited to the area having the above-described shape and size. For example, although the area determination unit 103 determines a rectangular input permission area in each of the above examples, it may determine a square, L-shaped, circular, elliptical, polygonal, or other input permission area. For example, for a user who draws a handwritten image from the left end to the right end of the display surface and then draws downward from that, the handwritten image is L-shaped, so the area determination unit 103 determines the L-shaped area. To do. The area determination unit 103 may determine the shape of the input permission area based on the above-mentioned history of handwritten input information.

Further, in the above example, when the horizontally written characters are drawn, the area determination unit 103 sets the horizontal size to the same size as the display surface, but the size may be smaller than that, or the shared screen scrolls. The size may be larger than that. The area determining unit 103 reduces the size of a user who tends to start a new line immediately and increases the size of a user who tends to handwrite to the edge of the screen based on, for example, the history of handwriting input information described above. .. Thereby, the area in which the user is more likely to perform handwriting input can be set as the input permission area.

[2-8] Image Showing Range of Input Allowed Area In each of the above examples, the range of the input allowed area is represented by displaying the image during writing, but it is not limited to this. For example, the screen data generation unit 104 may generate screen data in which an image showing the range of the input permission area is displayed. The image showing the range of the input permission area is, for example, an image of a line representing the boundary between the input permission area and other areas, or an image showing the background of the input permission area with a color or pattern different from that of other areas. Is. This allows the user to know to what extent the user is allowed to input.

[2-9] Category of Invention The present invention can be understood as an information processing system such as a screen sharing server device and a user device, as well as an information processing system such as a handwritten image sharing system including these devices. Further, the present invention can be regarded as an information processing method for realizing the processing executed by each device, and also as a program for causing a computer controlling each device to function. This program may be provided in the form of a recording medium such as an optical disc having the program stored therein, or may be provided in the form of being downloaded by a computer via a network such as the Internet and then installed and made available. May be done.

[2-10] Processing Procedures and the Like The processing procedures, sequences, flowcharts, and the like of the respective embodiments described in this specification may be interchanged as long as there is no contradiction. For example, the methods described herein present elements of the various steps in a sample order, and are not limited to the specific order presented.

[2-11] Handling of input/output information, etc. The input/output information, etc. may be stored in a specific place (for example, a memory) or may be managed by a management table. Information that is input/output may be overwritten, updated, or added. The output information and the like may be deleted. The input information and the like may be transmitted to another device.

[2-12] Software Software, whether called software, firmware, middleware, microcode, hardware description language, or any other name, is an instruction, instruction set, code, code segment, program code, program. , Subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., should be broadly construed.

Moreover, software, instructions, etc. may be transmitted and received via a transmission medium. For example, the software may use a wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and/or wireless technology such as infrared, wireless and microwave to websites, servers, or other When transmitted from a remote source, these wireline and/or wireless technologies are included within the definition of transmission medium.

[2-13] Information, Signals The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, instructions, commands, information, signals, bits, symbols, chips, etc., that may be mentioned throughout the above description are voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any of these. May be represented by a combination of

[2-14] System, Network As used herein, the terms "system" and "network" are used interchangeably.

[2-15] Meaning of “based on” As used herein, the phrase “based on” does not mean “based only on,” unless expressly specified otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[2-16] "and", "or"
In the present specification, with respect to a configuration that can be implemented with “A and B” or “A or B”, the configuration described in one expression may be used as the configuration described in the other expression. For example, when "A and B" is described, it may be used as "A or B" as long as it is practicable without inconsistency with other descriptions.

For example, in the above-described configuration in which the area determination unit 103 determines the “shape and size” of the input permission area for each user device 20, the area determination unit 103 determines the “shape or size” of the input permission area for each user. The configuration may be such that each device 20 is determined. In that case, not only the case where the area determination unit 103 determines the “whichever is selected among the shape and the size” of the input permitted area for each user device 20, but the area determination unit 103 does not simply determine the input permitted area. This includes a case where the “shape” is determined for each user device 20, and a case where the region determination unit 103 determines the “size” of the input permission region for each user device 20.

Further, for example, the configuration in which the area determining unit 103 determines the “shape” of the input permission area for each user device 20 does not exclude the configuration in which the shape other than the “shape” of the input permission area is also determined. Therefore, in the configuration in which the area determination unit 103 determines the “shape” of the input permission area for each user device 20, the area determination unit 103 determines the “shape and size” of the input permission area for each user device 20. The configuration of doing is also included. Thus, the description "A or B" also includes the embodiments "A and B" as long as it is practicable.

[2-17] Variations of Aspect, etc. The embodiments described in the present specification may be used alone, in combination, or may be switched according to execution. Further, the notification of the predetermined information (for example, the notification of “being X”) is not limited to the explicit notification, and is performed implicitly (for example, the notification of the predetermined information is not performed). Good.

Although the present invention has been described in detail above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the claims. Therefore, the description of the present specification is for the purpose of exemplifying explanation, and does not have any restrictive meaning to the present invention.

DESCRIPTION OF SYMBOLS 1... Handwriting image sharing system, 10... Screen sharing server device, 20... User device, 101... Handwriting input information acquisition part, 102... Character recognition part, 103... Area determination part, 104... Screen data generation part, 105... Screen data Output unit 106... Information storage unit 107... Handwriting image estimation unit 201... Handwriting input operation reception unit 202... Handwriting input information output unit 203... Handwriting input screen display unit.

Claims (7)

An information acquisition unit for acquiring input information of user's handwriting input from a plurality of user devices;
When the first input information is acquired from the first user device, the first handwritten image indicated by the first input information and the second handwritten image indicated by the second input information subsequent to the first input information are arranged. The first area is determined on the shared screen before the second input information is acquired, and the third input information is acquired from the second user device after the first input information is acquired from the first user device. When acquired, it is the second area in which the third handwritten image indicated by the third input information and the fourth handwritten image indicated by the fourth input information subsequent to the third input information are arranged, An area determination unit that determines a second area that does not overlap one area in the shared screen before the fourth input information is acquired;
By arranging the first handwritten image and the second handwritten image in the first area, and arranging the third handwritten image and the fourth handwritten image in the second area, screen data representing the shared screen is displayed. A generator to generate,
An information processing apparatus, comprising: an output unit that outputs the generated screen data. The information processing apparatus according to claim 1, wherein the area determination unit sets an area having a size or shape corresponding to the first handwritten image as the first area. A recognition unit for recognizing the character when the first handwritten image is the character,
The said area|region determination part makes the said area | region the size or shape according to the size of the said character, or the direction in which the said character arranges, when the said 1st handwritten image is recognized as a said 1st area | region. Information processing equipment. Equipped with a storage unit that stores the history of acquired input information ,
The area determination unit sets the area having a size or shape corresponding to a handwritten image estimated to be input from the history accumulated for the first user device as the first area. The information processing apparatus according to item 1. When the first handwritten image does not fit in the first area, the area determination unit expands the first area to a size in which the first handwritten image fits and expands the second area. The information processing apparatus according to claim 1, wherein the information processing apparatus is moved to a position that does not overlap the area. When the input of at least one of the first handwritten image and the third handwritten image has been completed and a gap has occurred between both images, the area determination unit is configured to fill the gap with the first handwritten image. The information processing apparatus according to claim 1, wherein one or both of the third handwritten image and the third handwritten image are moved. An image acquisition unit for acquiring a designated image designated by the user from the first user device;
The information processing apparatus according to claim 1, wherein the generation unit generates the screen data in which the acquired designated image is arranged and displayed in the first area.

Images