JP2018049503A - Information processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent handwritten images of respective users from overlapping even though the respective users input the handwritten images at the same time to display the handwritten images on a shared screen.SOLUTION: A handwriting input information acquisition part 101 acquires user's handwriting input information from a plurality of user devices. A character recognition part 102 recognizes a character in the case that a handwritten image shown by the acquired handwriting input information shows the character. An area determination part 103 respectively determines the shape and size of an area (an input permission area) where a handwriting input by each user device 20 is permitted on a shared screen about each user device 20. A screen data generation part 104 generates screen data representing a handwritten image shown by each handwriting input information acquired from the plurality of user devices 20, shown on the shared screen. A screen data output part 105 outputs the generated screen data.SELECTED DRAWING: Figure 4

Description

  The present invention relates to a technique for displaying a handwritten image on a shared screen.

  There is a technique for displaying a handwritten image on a shared screen. For example, in Patent Document 1, when handwritten input information input from its own terminal and handwritten input information transmitted from another terminal are displayed on the same display surface (that is, a shared screen), the handwritten input information is transmitted. A technique for displaying in a display color associated with a terminal that has been used is disclosed.

JP 2010-157098 A

In the technique of Patent Document 1, the handwritten image of another user is displayed in a color different from the handwritten image of his / her own. However, for example, a plurality of users input the same region at the same time, so the handwritten images overlap and become difficult to see. This is not expected.
Therefore, an object of the present invention is to prevent the handwritten images of those users from overlapping each other even if each user inputs simultaneously to display the handwritten images on a shared screen.

  In order to achieve the above-described object, an information acquisition unit that acquires information on user handwriting input from a plurality of user devices, and a handwritten image indicated by each of the information acquired from the plurality of user devices are displayed on a shared screen A generation unit for generating screen data, wherein the first handwritten image indicated by the information acquired from the first user device is arranged in a first area where handwriting input by the first user device is permitted on the shared screen. The second handwritten image indicated by the information acquired from the second user device during a period in which the information is acquired from the first user device does not overlap the first region on the shared screen and the first Information that includes a generation unit that generates the screen data by arranging in a second region where handwriting input by two user devices is permitted, and an output unit that outputs the generated screen data To provide a management apparatus.

Moreover, the said production | generation part is good also considering the area | region of the size or shape according to the information of the said handwritten input acquired from the said 1st user apparatus as said 1st area | region.
Furthermore, the recognition part which recognizes the said character when the said 1st handwritten image is a character is provided, The said production | generation part has the magnitude | size of the said character or the said character, when the said 1st handwritten image is recognized as a character. The line-up direction may be used as information for the handwriting input.

In addition, a storage unit that stores the history of the acquired handwritten input information is provided, and the generation unit responds to a handwritten image estimated to be input from the history stored for the first user device. A region of size or shape may be the first region.
Further, when the first handwritten image does not fit in the first area, the generation unit enlarges the first area to a size that fits the first handwritten image and enlarges the second area. You may move to the position which does not overlap with 1 area | region.

In addition, when the input of at least one of the first handwritten image and the second handwritten image is completed and a gap is generated between both images, the generating unit is configured to fill the first handwritten at a position where the gap is filled. One or both of the image and the second handwritten image may be moved.
Furthermore, the image acquisition part which acquires the designated image which the user designated from the said 1st user apparatus is provided, The said production | generation part produces | generates the said screen data which arranged and represented the acquired said designated image in the said 1st area | region. May be.

  ADVANTAGE OF THE INVENTION According to this invention, even if it inputs simultaneously in order that each user may display a handwritten image on a shared screen, it can prevent those users' handwritten images from overlapping.

The figure showing the whole structure of the handwritten image sharing system which concerns on an Example. The figure showing the hardware constitutions of a screen sharing server device The figure showing the hardware constitutions of a user apparatus The figure showing the functional composition which a handwritten image sharing system realizes The figure showing an example of the specification method of an indication position The figure showing an example of the determination method of an input permission area The figure showing an example of the handwritten image displayed on another user apparatus The figure showing another example of the determination method of an input permission area | region The figure showing an example of the operation procedure of each apparatus in shared display processing The figure showing an example of the enlarged input permission area The figure showing an example of the reduced handwritten image The figure showing an example of the handwritten image moved so that a clearance gap may be reduced The figure showing the functional structure which the handwritten image sharing system of a modification implement | achieves A figure showing an example of the accumulated history The figure showing the functional structure which the handwritten image sharing system of a modification implement | achieves A figure showing an example of the displayed designated image The figure showing an example of the whole structure of the handwritten image sharing system of a modification The figure showing the functional composition which a handwritten image sharing system realizes

[1] Embodiment FIG. 1 shows an overall configuration of a handwritten image sharing system 1 according to an embodiment. When the handwritten image sharing system 1 displays a shared screen on user devices respectively used by a plurality of users and the user performs handwriting input on the shared screen, not only the shared screen of the user device but also the user devices of other users This is a system in which a handwritten image input by handwriting is displayed on the sharing screen, and the handwritten image input by each user is shared by a plurality of users.

  The handwritten image sharing system 1 includes a network 2, a screen sharing server device 10, and user devices 20-1, 20-2 and 20-3 (hereinafter referred to as “user device 20” unless otherwise distinguished). The network 2 is a communication system including a mobile communication network and the Internet, and mediates exchange of data between devices accessing the own system. The screen sharing server apparatus 10 is accessing the network 2 via wired communication, and the user apparatus 20 is accessing via wireless communication. Access to the network 2 may be either wired communication or wireless communication.

  The screen sharing server device 10 is an information processing device that performs processing for sharing a screen for handwriting input displayed by each user device 20. The user device 20 displays a screen for handwriting input shared by the screen sharing server device 10, that is, a screen for sharing a handwritten image, accepts handwritten input by the user, and creates a handwritten image drawn by handwriting input (the device itself). It is an information processing apparatus which displays a user's handwritten image to be used, and a user's handwritten image using the other user apparatus 20).

  FIG. 2 shows a hardware configuration of the screen sharing server apparatus 10. The screen sharing server device 10 is a computer that includes a processor 11, a memory 12, a storage 13, a communication device 14, an input device 15, an output device 16, and a bus 17. Note that the term “apparatus” here can be read as a circuit, a device, a unit, or the like. Each device may include one or a plurality of devices, or some of the devices may not be included.

  The processor 11 controls the entire computer by operating an operating system, for example. The processor 11 may be configured by a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic device, a register, and the like. Further, the processor 11 reads programs (program codes), software modules, data, and the like from the storage 13 and / or the communication device 14 to the memory 12, and executes various processes according to these. The number of processors 11 that execute various processes may be one, two or more, and the two or more processors 11 may execute various processes simultaneously or sequentially. Further, the processor 11 may be implemented by one or more chips. The program may be transmitted from the network via a telecommunication line.

  The memory 12 is a computer-readable recording medium, and includes, for example, at least one of ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), and the like. May be. The memory 12 may be called a register, a cache, a main memory (main storage device), or the like. The memory 12 can store the above-described program (program code), software module, data, and the like.

  The storage 13 is a computer-readable recording medium such as an optical disc such as a CD-ROM (Compact Disc ROM), a hard disc drive, a flexible disc, a magneto-optical disc (eg, a compact disc, a digital versatile disc, a Blu- ray (registered trademark) disk, smart card, flash memory (eg, card, stick, key drive), floppy (registered trademark) disk, magnetic strip, or the like. The storage 13 may also be called an auxiliary storage device. The above-described storage medium may be, for example, a database including the memory 12 and / or the storage 13, a server, or other suitable medium. The storage 13 stores a program of an application (handwriting application) that realizes handwritten input.

  The communication device 14 is hardware (transmission / reception device) for performing communication between computers via a wired and / or wireless network, and is also referred to as, for example, a network device, a network controller, a network card, or a communication module. The input device 15 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that accepts an input from the outside.

  The output device 16 is an output device (for example, a display, a speaker, an LED lamp, or the like) that performs output to the outside. Note that the input device 15 and the output device 16 may have an integrated configuration (for example, a touch screen). The devices such as the processor 11 and the memory 12 are accessible to each other via a bus 17 for communicating information. The bus 17 may be composed of a single bus or may be composed of different buses between devices.

  The screen sharing server device 10 includes hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), and a field programmable gate array (FPGA). Hardware may be configured, and a part or all of each functional block may be realized by the hardware. For example, the processor 11 may be implemented by at least one of these hardware.

  FIG. 3 shows a hardware configuration of the user device 20. The user device 20 is a computer including a processor 21, a memory 22, a storage 23, a communication device 24, an input / output device 25, and a bus 26. The communication device 24 and the bus 26 from the processor 21 are hardware common to the devices of the same name shown in FIG. The input / output device 25 includes an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, and a touch sensor) that receives input from the outside, and an output device (for example, a display, a speaker, and an LED) that performs output to the outside. Lamp). In this embodiment, the user apparatus 20 includes a touch screen that is an input / output device in which an input device and an output device are integrated.

The functions described below are realized by the processor 11 of the screen sharing server device 10 and the processor 21 of each user device 20 executing a program to control each unit of each device.
FIG. 4 shows a functional configuration realized by the handwritten image sharing system 1. The screen sharing server apparatus 10 includes a handwritten input information acquisition unit 101, a character recognition unit 102, an area determination unit 103, a screen data generation unit 104, and a screen data output unit 105. The user device 20 includes a handwriting input operation reception unit 201, a handwriting input information output unit 202, and a handwriting input screen display unit 203. In FIG. 4, only one user device 20 is shown, but it is assumed that all user devices 20 have the same configuration.

  The handwriting input operation accepting unit 201 of the user device 20 accepts a handwriting input operation performed by the user indicating a position on the display surface. When the handwriting input operation reception unit 201 receives a handwriting input operation, the position (instructed position) on the display surface instructed by the operation is specified by, for example, the following method.

  FIG. 5 shows an example of a method for specifying the designated position. FIG. 5A shows a state in which the user's finger B1 indicates the position on the display surface A1 displaying the handwriting input screen. For example, as illustrated in FIG. 5B, the handwriting input operation reception unit 201 specifies the contact area F1 in contact with the display surface A1 of the finger B1, and specifies the center of the contact area F1 as the designated position G1. To do. Note that the method of specifying the designated position is not limited to this, and other known techniques may be used.

  The handwriting input operation reception unit 201 supplies position information indicating the specified pointing position to the handwriting input information output unit 202. Since the display surface A1 is rectangular as shown in the figure, the handwriting input operation accepting unit 201 has a certain corner of the display surface A1 (in this embodiment, the upper left corner when the upper side of the displayed character is turned upward). Is the origin P0, the coordinates of the designated position G1 in the coordinate system with one side forming the corner as the X axis and the other side as the Y axis are supplied as position information.

  In this coordinate system, the arrow direction (rightward direction) indicating the X axis in the figure is the positive direction of the X axis, and the arrow direction (downward direction) indicating the Y axis is the positive direction of the Y axis. . The handwriting input operation reception unit 201 repeatedly specifies the designated position and supplies the position information, for example, at regular time intervals. It is desirable that the time interval used here be narrow enough to recognize an image (a handwritten image such as a character or a picture) represented by the locus of the designated position.

  The handwritten input information output unit 202 outputs the information of the handwritten input received by the handwriting input operation receiving unit 201 to the screen sharing server device 10. Each time handwritten input information output unit 202 is supplied with position information repeatedly supplied from handwriting input operation receiving unit 201, that is, position information indicating a plurality of designated positions specified by handwriting input operation receiving unit 201. Output as input information. The handwritten input information is information representing a handwritten image (such as a character or a picture) input by handwriting, and a plurality of position trajectories indicated by the plurality of position information sequentially output by the handwritten input information output unit 202 are: Represents a handwritten image.

  The handwritten input information acquisition unit 101 of the screen sharing server device 10 acquires information on a user's handwritten input from a plurality of user devices. The handwritten input information acquisition unit 101 is an example of the “information acquisition unit” in the present invention. In the present embodiment, the handwritten input information acquisition unit 101 uses the above-described position information output by the user devices 20-1, 20-2, and 20-3 shown in FIG. Obtain as information. Each time handwritten input information acquisition unit 101 acquires handwritten input information, it supplies the acquired information to character recognition unit 102 and region determination unit 103.

  When the handwritten image indicated by the handwritten input information acquired by the handwritten input information acquiring unit 101 represents a character, the character recognizing unit 102 recognizes the character. The character recognition unit 102 is an example of the “recognition unit” in the present invention. The character recognition unit 102 uses, for example, OCR (Optical character recognition) technology based on handwritten input information supplied from the handwritten input information acquisition unit 101, that is, position information indicating an instruction position indicated by the user. The character indicated by the locus of the indicated position is recognized.

  When the character recognition unit 102 recognizes the character, the character recognition unit 102 supplies information indicating that the character has been recognized and the supplied handwritten input information to the region determination unit 103. When the character recognizing unit 102 does not recognize the character, the character recognizing unit 102 supplies information notifying that effect and the supplied handwritten input information to the region determining unit 103. In this embodiment, a case where a character is recognized will be described.

The area determination unit 103 determines the shape and size of an area (input permission area) in which handwriting input by each user apparatus 20 is permitted on the shared screen for each user apparatus 20. A method of determining the input permission area will be described with reference to FIGS.
FIG. 6 shows an example of a method for determining the input permission area. In FIG. 6 (a), the user 1 to use the user device 20-1, display the handwriting input screen on the display surface A1, was entered handwritten image B11 that "Hello!" Is shown. The handwritten image B11 is displayed by operations of a screen data generation unit 104, a screen data output unit 105, and a handwriting input screen display unit 203, which will be described later, which will be described later.

  When the information indicating that the character has been recognized is supplied, the region determination unit 103 supplies the handwritten image indicated by the supplied handwritten input information, that is, the rectangle C1 circumscribing the handwritten character (each side is shown in FIG. 5). FIG. 6B shows the side length L11 along the X axis and the side length L12 along the Y axis of the coordinate system centered on the origin P0 along the X axis and the Y axis. To calculate. The region determination unit 103 determines that the handwritten character is drawn in horizontal writing if L11> L12 and the handwritten character is drawn in vertical writing if L11 <L12.

  In this example, the area determination unit 103 determines that the handwritten character is horizontal writing. In this case, the region determination unit 103 determines a horizontally long rectangle as the input permission region. Specifically, the region determination unit 103 extends the upper side of the quadrangle C1 to the left and right to the end of the display surface A1, and increases the size in the vertical direction at a predetermined rate than the quadrangle C1 (in this example, N (N Is a natural number) multiplied by a square) is determined as the input permission area D1. In this embodiment, the horizontal determination character is written up to the right end and the next character is written on the lower side when a new line is entered, and the area determination unit 103 inputs the input permission area so that the area extends below the input handwritten character. To decide.

  When the area determination unit 103 determines that the handwritten character is vertical writing, the area determination unit 103 determines a vertically long rectangle as the input permission area. In that case, the area determination unit 103 extends a right side of the quadrilateral circumscribing the handwritten character up and down to the end of the display surface A1, and forms a quadrangle whose horizontal size is larger than the quadrangle by a predetermined ratio. Determine as. In this embodiment, assuming that the vertically written characters are written to the bottom and a new line is written, the next character is written on the left side, and the area determination unit 103 sets the input permission area so that the area extends to the left side of the input handwritten characters. decide.

  Since the user 1 is allowed to input handwriting in the input permission area D1 determined in the example of FIG. 6, the handwritten image input by the user 1 in the input permission area D1 is the handwritten image B12 shown in FIG. Is displayed. On the other hand, handwriting input to the input permission area D1 is not permitted for other users. Therefore, for example, hand-drawn image user 2 to use the user device 20-2 has handwriting input is displayed in the region outside the input permission area D1 as handwritten image B21 that "Hello!" Depicted in Fig. 6 (e) Will be.

  In FIG. 6 (e), a writing image H1 indicating that another user is writing is displayed at the lower right of the handwritten image B21. This in-writing image H1 represents the range of the input permission area permitted by the user 2. In this example, an image of a handwritten character string prepared in advance is used as the writing image H1 instead of an image handwritten by the user 2.

  FIG. 7 shows an example of a handwritten image displayed on another user device. In FIG. 7 (a), the display surface A2 of the user device 20-2, the hand-drawn image B11 is displayed that is handwriting input at the user device 20-1 "Hello!". In addition, a writing image H2 is displayed on the display surface A2. This in-writing image H2 is displayed at the lower end of the input permission area D1 described in the description of FIG. 6, and represents the lower end of the input permission area D1. Since the handwritten image is not displayed even if it is input to the region sandwiched between the handwritten image B11 and the writing image H2, the user 2 starts handwriting input from below the writing image H2.

  In FIG.7 (b), the handwritten image B21 which the user 2 input into the area | region outside the input permission area | region D1 is displayed on the display surface A2. Since the handwritten image B21 is recognized as a character by the character recognizing unit 102, the region determining unit 103 determines the length L21 of the side along the X axis of the quadrangle C2 circumscribing the handwritten image B21, as in the example of FIG. The side length L22 along the Y-axis is calculated as shown in FIG.

  The area determination unit 103 determines that the character represented by the handwritten image B21 is horizontal writing because L21> L22, and similarly to the input permission area D1, based on the rectangle C2, the input permission area D2 (the vertical size is N times L22). Are determined as shown in FIG. 7 (d). At this time, the writing image H1 displayed on the display surface A1 in FIG. 6E is not displayed on the display surface A2.

Further, the area determination unit 103 determines an input permission area based on handwritten input information supplied from the handwritten input information acquisition unit 101 before the character recognition unit 102 recognizes a character.
FIG. 8 shows another example of the method of determining the input permission area. FIG. 8A shows that the user 2 has designated the designated position G2 on the display surface A2 of the user device 20-2 and started handwriting input. For example, the region determination unit 103 has the indicated position G2 indicated by the handwritten input information at the center in the vertical direction, and the vertical size is determined in advance (in this example, the size L1 is determined). A rectangle having the horizontal width of the display surface as the horizontal size is determined as the input permission area D3.

  At this time, as shown in FIG. 8B, only the writing image H3 is displayed in the input permission area D3 and the handwritten image input by the user 2 is not yet displayed on the display surface A1 of the user device 20-1. . Also in this case, the writing image H3 represents the range of the input permission area permitted by the user 2. The area determination unit 103 supplies area information indicating the input permission area determined as described above to the screen data generation unit 104 together with the supplied handwriting input information.

  The screen data generation unit 104 generates screen data representing the supplied handwritten input information, that is, the handwritten image indicated by each handwritten input information acquired from the plurality of user devices 20 on the shared screen. Further, the screen data generation unit 104 is, for example, a user device 20-1 (an example of the “first user device” of the present invention) in which the user 1 performs handwriting input and the user device 20 is in a period during which the handwriting input is performed. -2 (an example of the “second user device” of the present invention), when the user 2 performs handwriting input, screen data is generated as follows.

  The screen data generation unit 104 arranges the handwritten image indicated by the handwritten input information acquired from the user device 20-1 in the first input permission area where the handwritten input by the user device 20-1 is permitted on the shared screen. Hereinafter, the handwritten image previously input in this manner is referred to as a “first handwritten image”, and the input permission area permitted for the user device 20 in which the first handwritten image is input is referred to as a “first input permission area”. . In the example of FIG. 6, the screen data generation unit 104 arranges the handwritten image B11 that is the first handwritten image, with the input permission area D1 determined by the area determination unit 103 as the first input permission area.

  In addition, the screen data generation unit 104 displays the first handwritten image indicated by the handwritten input information acquired from the user device 20-2 during the period in which the handwritten input information is acquired from the user device 20-1 on the shared screen. The screen data is generated by being arranged in an input permission area that does not overlap with the input permission area and in which handwriting input by the user device 20-2 is permitted. Hereinafter, the handwritten image input later is referred to as “second handwritten image”, and the input permission area permitted for the user device 20 to which the second handwritten image is input is referred to as “second input permission area”. .

  Here, the “period during which handwritten input information is acquired” means only the moment when the handwritten input information acquisition unit 101 acquires the information of the handwritten input (the moment when the data packet indicating the information of the handwritten input is received). Instead, the user 1 continues to perform handwriting input to the user device 20-1, and the entire period during which handwritten input information is intermittently acquired is assumed. For example, when the period during which handwritten input information is not acquired exceeds a threshold, the screen data generation unit 104 determines that the period during which handwritten input information is acquired has ended.

  In the example of FIG. 6 (d), the has been instructed to until the user 1 is completed the input of the handwritten image B12 of "Hello!", "It's good weather today!" From the start of the input of the handwritten image B11 that It is a period in which position information indicating the position is intermittently acquired as handwritten input information. In the example of FIG. 7, handwritten input information (position information indicating the trajectory of the handwritten image B <b> 12 that is “good weather today!”) Is acquired from the user device 20-1 during the period in which the handwritten input information is acquired. of handwriting input information ( "Hello!" position information indicating the trajectory of the handwritten image B21 that) has been acquired.

  In this case, the screen data generation unit 104 uses the input permission area D2 determined as an area that does not overlap the input permission area D1 that is the first input permission area as the second input permission area, and the handwritten image B21 that is the second handwritten image. Is arranged. In this embodiment, since the area determination unit 103 determines the input permission area as described in the description of FIG. 6B, the screen data generation unit 104 acquires the handwritten input information acquired from the user device 20-1. An area having a size and a shape corresponding to the input permission area (input permission area D1 in the example of FIG. 6B) is set.

  More specifically, when the handwritten image indicated by the handwritten input information acquired from the user device 20-1 is recognized as a character by the character recognizing unit 102, the screen data generating unit 104 displays the character (FIG. 6B). In the example of), an area having a size and a shape corresponding to the size of the handwritten image B11) and the direction in which the characters are arranged is set as the input permission area (the input permission area D1 in the example of FIG. 6B).

  The input permission area D1 is an area whose vertical size is larger than the vertical size of the handwritten image B11 by a predetermined ratio (N times in the example of FIG. 6). The input permission area D1 is a horizontally long rectangular area because the character represented by the handwritten image B11 is horizontally written. In addition, when the handwritten character is vertically written, the input permission region is a vertically long rectangular shape as described above, and the horizontal size is a region in which the horizontal size of the handwritten character is increased by a predetermined ratio. Become. The screen data generation unit 104 supplies the screen data generated as described above to the screen data output unit 105.

  The screen data output unit 105 outputs the screen data generated by the screen data generation unit 104. The screen data output unit 105 is an example of the “output unit” in the present invention. In the handwritten image sharing system 1, a group of user devices 20 that supply handwritten images is registered. The screen sharing server device 10 stores the destination (IP (Internet Protocol) address or the like) of the user device 20 in the same group in association with information (for example, group ID (Identification)) for identifying the group.

  In this embodiment, the user devices 20-1, 20-2, and 20-3 are registered as the same group. When the handwritten input information acquisition unit 101 acquires the handwritten input information from the user devices 20-1 and 20-2 as in the example of FIGS. 6 and 7, the screen data output unit 105 starts from the screen data generation unit 104. The supplied screen data is output to the user devices 20-1, 20-2, and 20-3.

  The handwriting input screen display unit 203 of the user device 20 displays the screen indicated by the screen data output from the screen sharing server device 10. The handwriting input screen display unit 203 of the user device 20-1 displays a screen, for example, as shown in FIG. 6, and the handwriting input screen display unit 203 of the user device 20-2, for example, displays a screen as shown in FIG. To do. The handwriting input screen display unit 203 of the user device 20-3 also displays the handwritten image input by the user 1 and the user 2 by handwriting. Moreover, if the user 3 using the user device 20-3 performs handwriting input, the handwriting input screen display unit 203 of each user device 20 displays the handwritten image.

The screen sharing server device 10 and each user device 20 perform shared display processing for displaying a user's handwritten image on the shared screen based on the above configuration.
FIG. 9 shows an example of the operation procedure of each device in the shared display processing. Although only the user devices 20-1 and 20-2 are shown in FIG. 9, the user device 20-3 is assumed to perform the same operation. This operation procedure is started, for example, when each user operates his / her user device 20 to activate a handwriting application and performs an operation of drawing a handwritten image.

  First, the user device 20 (handwriting input operation reception unit 201) receives a handwriting input operation performed by the user instructing a display surface (steps S11 and S21). Note that common operations of the user devices 20-1 and 20-2 are collectively expressed in this way. Next, the user device 20 (handwritten input information output unit 202) outputs information of the handwritten input whose operation is accepted to the screen sharing server device 10 (steps S12 and S22).

  The screen sharing server device 10 (handwriting input information acquisition unit 101) acquires the handwriting input information respectively output from the user devices 20-1 and 20-2 (steps S31 and S32). Next, if the handwritten image indicated by the acquired handwritten input information is a character, the screen sharing server device 10 (character recognition unit 102) recognizes the character (step S33). Subsequently, the screen sharing server apparatus 10 (area determination unit 103) determines an area having a size and a shape corresponding to the recognized character size and the direction in which the characters are arranged as an input permission area (step S34).

  Next, the screen sharing server device 10 (screen data generation unit 104) generates screen data indicating a shared screen in which a handwritten image is arranged in the determined input permission area (step S35). Subsequently, the screen sharing server device 10 (screen data output unit 105) outputs the generated screen data to each of the plurality of user devices 20 in the same group (steps S36 and S37). The user device 20 (handwriting input screen display unit 203) displays the shared screen indicated by the output screen data (steps S13 and S23). And the user apparatus 20 returns to step S11, S21, and continues operation | movement. In this way, the handwritten image input by the user by handwriting is displayed using the handwriting input screen of each user device 20 as a shared screen.

  In this embodiment, even if each user inputs simultaneously to display the handwritten image on the shared screen, the handwritten image B11 of the user 1 is displayed in the input permission area D1 as shown in the example of FIG. Since the handwritten image B21 is displayed in the input permission area D2 determined as an area that does not overlap the input permission area D1, the handwritten images of the users 1 and 2 can be prevented from overlapping.

  In this embodiment, when the user inputs a handwritten character, an area having a size corresponding to the size of the character is an input permission area. Thus, for example, a large input permission area is secured for a user who writes a large character, and a small input permission area is secured for a user who writes a small character, compared to a case where the size of the input permission area is constant, The difference between users in the number of characters that can be entered in the input permission area can be reduced.

  In this embodiment, when a user inputs handwritten characters, an area having a shape corresponding to the direction in which the characters are arranged is an input permission area. As a result, for example, a user who writes characters in horizontal writing has a rectangular input permission area that is long horizontally, and a user who writes characters in vertical writing has a long vertical input permission area. Compared to the case where the shape is constant, it is possible to prevent the need for line breaks immediately after writing and making it difficult to write.

[2] Modifications The above-described embodiments are merely examples of the present invention, and may be modified as follows. Moreover, you may implement an Example and each modification in combination as needed.

[2-1] Expansion of input permission area There are cases where the handwritten image of the user does not fit in the input permission area secured as described in the embodiment. In this case, the input permission area may be enlarged so that the handwritten image is accommodated. In the present modification, when the handwritten image does not fit in the input permission area, the area determination unit 103 determines an input permission area that has been enlarged to a size that allows the handwritten image to fit.

Therefore, when the handwritten image does not fit in the input permission area, the screen data generation unit 104 also generates screen data indicating a screen in which the input permission area is enlarged to fit the handwritten image.
FIG. 10 shows an example of the enlarged input permission area. Figure 10 (a), the are displayed hand-drawn image B13 that the user 1 has handwriting input "Hello! It's good weather today!" To the input enable area D4 of the display surface A1 of the user device 20-1, input and in the image H4 writing the handwritten image B24 that the user 2 has handwriting input "Hello!" is on the lower side of the permit area D4 are displayed in the input permission area D5.

  In FIG. 10B, the handwritten image B14 “DO” which the user 1 inputs by handwriting continuously below the handwritten image B13 is displayed on the display surface A1. Since the handwritten image B14 is not stored in the input permission area D4, the screen data generation unit 104 enlarges the input permission area D4 to a size that allows the handwritten image B14 to be stored, and forms the input permission area D6. Screen data on which the image B14 is arranged is generated.

  Further, the screen data generation unit 104 is configured such that the input permission area D5 in which the handwritten image B24 displayed in the enlarged portion is arranged does not overlap the input permission area D6 because the input permission area is enlarged to D6. Has been moved to. In this way, the screen data generation unit 104 does not fit the first handwritten image (the handwritten image input first. In this example, the handwritten image B14) does not fit in the first input permission area (in this example, the input permission area D4). In this case, the first input permission area is enlarged to a size in which the first handwritten image can be accommodated, and the second input permission area (the handwritten image input later, handwritten image B24 in this example) is arranged ( In this example, the input permission area D5) is moved to a position that does not overlap the enlarged first input permission area.

  In this modified example, the input permission area is enlarged to D6 and the input permission area D5 is moved (the handwritten image B24 and the writing image H4 arranged in the input permission area D5 are also moved). As shown in FIG. 10C, the handwritten image B15 “Let's go play somewhere!” Can be drawn without overlapping the handwritten image B24 of the user 2.

  In addition, the input permission area | region determined before the character described in the example of FIG. 8 is recognized will be expanded to a larger input permission area | region, if a character is recognized. If this input permission area is set as the first input permission area, the screen data generation unit 104 also does not overlap with the enlarged first input permission area in this case as well. What is necessary is just to move to a position. Thereby, even if the 2nd handwritten image which the user 2 started drawing before the 1st handwritten character of the user 1 is recognized is displayed, the user 1 does not superimpose the 1st handwritten character on the 2nd handwritten character. I can continue to write.

[2-2] Reduction of Handwritten Image When the user's handwritten image does not fit in the input permission area as described above, the handwritten image may be reduced instead of expanding the input permission area. In the present modification, when the handwritten image does not fit in the input permission area, the screen data generation unit 104 generates screen data indicating a screen in which the handwritten image is reduced and arranged in a size that fits in the input permission area.

  FIG. 11 shows an example of a reduced handwritten image. In the example of FIG. 11, the screen data generation unit 104 generates screen data indicating a screen in which the handwritten images B13 and B15 reduced so as to fit in the input permission area D4 shown in FIG. 10 are arranged in the input permission area D4. . In this modification, when the handwritten image does not fit in the input permission area, the handwritten image can be continuously drawn without being superimposed on the handwritten image of the other user without moving the handwritten image of the other user.

[2-3] Reduction of gap If the handwritten image input to the secured input permission area is small, the input permission area that has not been input remains as a gap with the handwritten image of another user. An operation for reducing the gap may be performed.

  In the present modification, when the screen data generation unit 104 completes input of at least one of the first handwritten image and the second handwritten image and a gap is generated between the two images, the screen data generation unit 104 is placed at a position where the gap is filled. Screen data indicating a screen on which one or both of the first handwritten image and the second handwritten image are moved is generated.

  FIG. 12 shows an example of the handwritten image moved so as to reduce the gap. Figure 12 (a), the display surface A2 of the user device 20-2, represents the handwritten image B15 that the user 1 has handwriting input "Hello!", The range of the input enable area D7 where the handwritten image B15 is arranged A writing image H5 is displayed. In addition, the hand-drawn image B17 of "Hello!" User 2 handwriting input to the input enabled area D8, which is secured on the lower side of the input enable area D7 is displayed.

  In this example, the user 2 continues the handwriting input after the state of FIG. 12A, but the user 1 has stopped the handwriting input, so the in-writing image H5 is displayed as shown in FIG. 12B. It is no longer being done. Therefore, between the handwritten image B15 and the handwritten image B17, a gap E1 having a length L31 in the vertical direction is generated as shown in FIG. In this case, since the screen data generation unit 104 has completed the input of the handwritten image B15 and a gap has occurred in the handwritten images B15 and B17, a screen showing a screen in which the handwritten image B15 is moved to a position to fill the gap. Generate data.

  The screen indicated by the screen data generated in this way is displayed on the display surface A2 as shown in FIG. In this example, the handwritten images B15 and B17 are displayed separated by a gap E2 (the length in the vertical direction is L32 (L32 <L31)). As a result, the gap between the handwritten images displayed on the shared screen can be made smaller than when the gap is not reduced, and more handwritten images are displayed on the shared screen. Can do.

[2-4] Method for Determining Input Permission Area The area determination unit 103 may determine the input permission area by a method different from the above example.
FIG. 13 shows a functional configuration realized by the handwritten image sharing system 1a of the present modification. The handwritten image sharing system 1a includes a screen sharing server device 10a and a user device 20. The screen sharing server device 10a includes an information storage unit 106 and a handwritten image estimation unit 107 in addition to the units shown in FIG.

  In this modification, when the character recognizing unit 102 recognizes a character, the character recognizing unit 102 supplies information indicating that the character is recognized and the supplied handwritten input information to the information accumulating unit 106 and may not recognize the character. The supplied handwritten input information is supplied to the information storage unit 106. The information storage unit 106 stores the supplied handwritten input information, that is, the history of the handwritten input information acquired by the handwritten input information acquisition unit 101.

  FIG. 14 shows an example of the accumulated history. In the example of FIG. 14, the information storage unit 106 inputs the type of handwritten image, the size of the handwritten image, and the input for each user device that uses the user device 20 (the user devices 20-1 and 20-2 in this example). The date and time are stored in association with each other. In this modified example, when the handwritten input information output unit 202 of the user device 20 outputs handwritten input information, for example, the device ID is output together as identification information for identifying the own device, and the device ID is the handwritten input information acquisition unit. 101 and the character recognition unit 102 are supplied to the information storage unit 106 (the device IDs of the user devices 20-1 and 20-2 are “U001” and “U002”, respectively).

  Similar to the screen data generation unit 104, the information storage unit 106 determines a period during which handwritten input information is acquired, and determines the type and size of the handwritten image based on the handwritten input information supplied during that period. . If the character is recognized by the character recognition unit 102, the information storage unit 106 determines whether horizontal writing is vertical writing as in the area determination unit 103 described in the explanation of FIG. For example, “horizontal writing character” is stored as a type of handwritten image. Further, the information storage unit 106 stores “picture” as the type of handwritten image if the character recognition unit 102 has not recognized the character.

  Further, the information storage unit 106 stores the size of a rectangle circumscribing the handwritten image indicated by the handwritten input information as the size of the handwritten image. Further, the information storage unit 106 stores, for example, the date and time when the period during which handwritten input information is acquired starts or ends as the input date and time. The information storage unit 106 stores these pieces of information in association with the device ID supplied together with the handwritten input information.

  The handwritten image estimation unit 107 estimates a handwritten image estimated to be input by the user from the history of handwritten input accumulated for the user device 20 used by the user. The handwritten image estimation unit 107 is, for example, handwritten that is estimated to be input by the user using the user device 20 corresponding to the device ID, for example, the type of handwritten image that is stored most frequently in association with the device ID. Estimated as an image.

  For example, for the user device 20-1 with the device ID “U001”, the handwritten image estimation unit 107 estimates that “horizontal characters” are drawn because the types of handwritten images are “horizontal characters”. The handwritten image estimation unit 107 estimates that “picture” is drawn for the user device 20-2 with the device ID “U002” because “vertical writing” is once and “picture” is twice. To do. The handwritten image estimation unit 107 supplies information indicating the handwritten image thus estimated (type of handwritten information) to the region determination unit 103.

  The area determination unit 103 inputs an area having a size and a shape according to the type of the supplied handwritten image, that is, the handwritten image estimated to be input from the handwriting input history accumulated for the user device 20. Determine as. For example, when “horizontal writing characters” are estimated, the region determination unit 103 determines a rectangle having a full width in the horizontal direction of the display surface as the shape of the input permission region. Then, the region determination unit 103 calculates the average value of the vertical sizes of the handwritten images stored in association with “horizontal writing characters”, and determines the average value as the vertical size.

  In the example of FIG. 14, the area determination unit 103 determines (L41 + L43 + L45) ÷ 3 based on the vertical size of the handwritten image drawn as horizontally written characters for the user device 20-1 with the device ID “U001”. A rectangle having a vertical size of is determined as the input permission area. In addition, for example, when “vertically written characters” are estimated, the region determining unit 103 determines a rectangle having a full width in the vertical direction of the display surface as the shape of the input-permitted region, and the handwriting drawn as vertically written characters A rectangle whose horizontal size is the average value of the horizontal size of the image is determined as the input permission area.

  In addition, when the “picture” is estimated, the area determination unit 103 calculates the average value of the vertical size and the average value of the horizontal size of the handwritten image drawn as a picture, and calculates the average value thereof. A rectangle having a vertical size and a horizontal size is determined as an input-permitted area. In the example of FIG. 14, for the user device 20-2 whose device ID is “U002”, the region determination unit 103 vertically calculates the average value (L53 + L55) ÷ 2 of the vertical size of the handwritten image drawn as a picture. A rectangle having the size in the direction and the average value in the horizontal direction (L54 + L56) ÷ 2 in the size in the horizontal direction is determined as the input permission area.

  Since the area determination unit 103 determines the input permission area as described above, the screen data generation unit 104 has a size corresponding to the handwritten image estimated to be input from the handwriting input history accumulated for the user device 20 and The shape area is set as the input permission area. Thereby, even if it is before a character is recognized by the character recognition part 102 after the handwriting input information acquisition part 101 starts acquisition of the information of handwriting input like the example of FIG. 8, the tendency of a user's handwriting input It is possible to secure an area suitable for the input permission area.

  In the above example, the average value of the sizes is calculated. However, the present invention is not limited to this, and the maximum value, the minimum value, or the intermediate value of the sizes may be calculated. Further, the history used for estimation may be limited to only those accumulated in a specific period (for example, the past year). In this case, an area that matches the recent tendency of handwriting input by the user can be secured as the input permission area.

[2-5] Display of non-handwritten image An image other than a handwritten image (an image drawn by an operation received by the handwriting input operation receiving unit 201 of the user device) may be displayed in the input permission area.
FIG. 15 illustrates a functional configuration realized by the handwritten image sharing system 1b according to the present modification. The handwritten image sharing system 1b includes a screen sharing server device 10b and a user device 20b. The screen sharing server apparatus 10b includes a designated image acquisition unit 108 in addition to the units illustrated in FIG. The user device 20b includes a designated image output unit 204 in addition to the units illustrated in FIG.

  The designated image output unit 204 designates an image displayed on the display surface, displays a file management application or the like, designates an image file, and pastes the designated image in the input permission area (for example, being displayed) When an operation such as dragging and dropping the image or image file to the input-permitted area is received, the image designated by those operations is output to the screen sharing server apparatus 10b. The designated image obtaining unit 108 obtains the image output in this way, that is, the designated image designated by the user from the user device 20. The designated image acquisition unit 108 is an example of the “image acquisition unit” in the present invention.

The designated image acquisition unit 108 supplies the acquired designated image to the screen data generation unit 104. The screen data generation unit 104 generates screen data in which the designated image acquired by the specified image acquisition unit 108 is arranged and represented in the input permission area.
FIG. 16 shows an example of the displayed designated image. In the example of FIG. 16, the screen data generation unit 104 generates screen data represented by arranging the designated image J1 in the input permission area D1 shown in FIG. The designated image output unit 204 outputs position information indicating the pasting position where the designated image is pasted together with the designated image, so that the screen data generation unit 104 has placed the designated image at the pasting position of the input permission area. Data may be generated.

  In addition, when the designated image protrudes from the input permission area, the screen data generation unit 104 may generate screen data that is reduced to a size that fits the designated image in the input permission area, or to a size that fits the designated image. Screen data in which the input permission area is enlarged may be generated. In addition, as the designated image, characters represented by a font input with a keyboard or the like may be used. According to this modification, it is possible to display a designated image such as a photograph, a CG (Computer Graphics), and a character represented by a font together with the handwritten image on the handwritten image sharing screen.

[2-6] Apparatus for Implementing Each Unit The apparatus for implementing each unit illustrated in FIG. 4 is not limited to the above-described apparatus. For example, one or both functions of the character recognition unit 102 and the region determination unit 103 may be realized by another server device. In addition, the user device may realize the function realized by the screen sharing server device.

  FIG. 17 shows an example of the overall configuration of the handwritten image sharing system 1c according to this modification. The handwritten image sharing system 1c includes a network 2 and user devices 20c-1, 20c-2, and 20c-3 (hereinafter referred to as “user device 20c” unless otherwise distinguished). Each user device 20 c communicates with each other via the network 2. In addition, each user apparatus 20c communicates by P2P (Peer to Peer), or the other user apparatus 20c communicates through the user apparatus 20c because one user apparatus 20c functions as a wireless router. May be.

  FIG. 18 shows a functional configuration realized by the handwritten image sharing system 1c. The user device 20c includes various units included in the screen sharing server device in addition to the units included in the user device 20 in FIG. The handwritten input information acquisition unit 101 of the user device 20c acquires handwritten input information from the own device and the other user device 20c, and the screen data generation unit 104 generates screen data based on the acquired information on the handwritten input. Is generated. Further, the screen data output unit 105 of the user device 20c displays the shared screen by outputting the screen data to the handwriting input screen display unit 203 of the own device. According to this example, a handwritten image can be displayed on a shared screen of each user device and shared by a plurality of users without providing a server device.

[2-7] Input permission area The input permission area is not limited to the area having the shape and size described above. For example, in the above example, the area determination unit 103 has determined the rectangular input permission area, but may determine an input permission area having a shape such as a square, an L shape, a circle, an ellipse, or a polygon. For example, for a user with a habit of drawing a handwritten image from the left end to the right end of the display surface and then drawing downward from there, the handwritten image becomes L-shaped, so the region determination unit 103 determines the L-shaped region. To do. The area determination unit 103 only needs to determine the shape of the input permission area based on the above-described history of handwritten input information.

  In the above example, for example, when horizontally written characters are drawn, the area determination unit 103 sets the horizontal size to the same size as the display surface, but the size may be smaller than that, or the shared screen scrolls. In some cases, the size may be larger. For example, based on the above-described history of handwritten input information, the area determination unit 103 reduces the size of a user who tends to immediately start a new line, and increases the size of a user who tends to hand-draw to the end of the screen. . Thereby, an area | region where possibility that a user will perform handwriting input is higher can be made into an input permission area | region.

[2-8] Image Showing Range of Input Permitted Area In each of the above examples, the range of the input permitted area is represented by displaying the image being written, but this is not restrictive. For example, the screen data generation unit 104 may generate screen data representing an image indicating the range of the input permission area. The image indicating the range of the input-permitted area is, for example, an image of a line that represents the boundary between the input-permitted area and the other area, or an image that represents the background of the input-permitted area in a color or pattern different from the other areas. It is. As a result, the user can know how far the area is permitted to be input to the user.

[2-9] Category of Invention The present invention can be understood as an information processing system such as a handwritten image sharing system including such devices as well as information processing devices such as a screen sharing server device and a user device. In addition, the present invention can be understood as an information processing method for realizing processing performed by each device, or as a program for causing a computer that controls each device to function. This program may be provided in the form of a recording medium such as an optical disk in which it is stored, or may be provided in the form of being downloaded to a computer via a network such as the Internet, installed and made available for use. May be.

[2-10] Processing Procedures, etc. The processing procedures, sequences, flowcharts, and the like of each embodiment described in this specification may be switched in order as long as there is no contradiction. For example, the methods described herein present the elements of the various steps in an exemplary order and are not limited to the specific order presented.

[2-11] Handling of input / output information and the like Input / output information and the like may be stored in a specific location (for example, a memory) or may be managed by a management table. Input / output information and the like can be overwritten, updated, or additionally written. The output information or the like may be deleted. The input information or the like may be transmitted to another device.

[2-12] Software Software, regardless of whether it is called software, firmware, middleware, microcode, hardware description language, or another name, is an instruction, instruction set, code, code segment, program code, program , Subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, etc. should be interpreted broadly.

  Also, software, instructions, etc. may be transmitted / received via a transmission medium. For example, software may use websites, servers, or other devices using wired technology such as coaxial cable, fiber optic cable, twisted pair and digital subscriber line (DSL) and / or wireless technology such as infrared, wireless and microwave. When transmitted from a remote source, these wired and / or wireless technologies are included within the definition of transmission media.

[2-13] Information, Signals The information, signals, etc. described herein may be represented using any of a variety of different technologies. For example, data, commands, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description are voltages, currents, electromagnetic waves, magnetic fields or magnetic particles, light fields or photons, or any of these May be represented by a combination of

[2-14] System, Network As used herein, the terms “system” and “network” are used interchangeably.

[2-15] Meaning of “based on” As used herein, the term “based on” does not mean “based only on”, unless expressly specified otherwise. In other words, the phrase “based on” means both “based only on” and “based at least on.”

[2-16] “and”, “or”
In the present specification, for a configuration that can be implemented by either “A and B” or “A or B”, a configuration described in one expression may be used as a configuration described in the other expression. For example, when “A and B” is described, it may be used as “A or B” as long as it is feasible without inconsistency with other descriptions.

  For example, in the configuration in which the area determination unit 103 determines the “shape and size” of the input permission area for each user device 20, the area determination unit 103 determines the “shape or size” of the input permission area for each user. It is good also as a structure which determines each about the apparatus 20. FIG. In that case, not only the case where the area determination unit 103 determines “whichever shape or size is selected” of the input permission area for each user device 20, but the area determination unit 103 simply determines whether the input permission area The case where the “shape” is determined for each user device 20 and the case where the area determination unit 103 determines the “size” of the input permission area for each user device 20 are also included.

  In addition, for example, the configuration in which the area determination unit 103 determines the “shape” of the input permission area for each user device 20 does not exclude the configuration in which other than the “shape” of the input permission area is determined. Therefore, in the configuration in which the area determination unit 103 determines the “shape” of the input permission area for each user device 20, the area determination unit 103 determines the “shape and size” of the input permission area for each user device 20. The configuration of performing is also included. Thus, the description “A or B” includes the embodiments “A and B” as long as they are feasible.

[2-17] Variations of Aspect, etc. Each embodiment described in the present specification may be used alone, in combination, or may be switched according to execution. In addition, notification of predetermined information (for example, notification of being “X”) is not limited to explicitly performed, but is performed implicitly (for example, notification of the predetermined information is not performed). Also good.

  Although the present invention has been described in detail above, it will be apparent to those skilled in the art that the present invention is not limited to the embodiments described herein. The present invention can be implemented as modified and changed modes without departing from the spirit and scope of the present invention defined by the description of the scope of claims. Therefore, the description of the present specification is for illustrative purposes and does not have any limiting meaning to the present invention.

DESCRIPTION OF SYMBOLS 1 ... Handwritten image sharing system, 10 ... Screen sharing server apparatus, 20 ... User apparatus, 101 ... Handwritten input information acquisition part, 102 ... Character recognition part, 103 ... Area determination part, 104 ... Screen data generation part, 105 ... Screen data Output unit 106 ... information storage unit 107 107 handwritten image estimation unit 201 handwritten input operation accepting unit 202 handwritten input information output unit 203 handwritten input screen display unit

Claims (7)

An information acquisition unit for acquiring user handwriting input information from a plurality of user devices;
A generation unit that generates screen data representing a handwritten image indicated by each piece of information acquired from a plurality of user devices on a shared screen, the first handwritten image indicated by the information acquired from the first user device In the first area where handwriting input by the first user device is permitted on the shared screen, and the information acquired from the second user device during the period in which the information is acquired from the first user device A generation unit configured to generate the screen data by arranging the second handwritten image indicated by the second handwritten image in a second region that does not overlap the first region on the shared screen and is allowed to be handwritten by the second user device;
An information processing apparatus comprising: an output unit that outputs the generated screen data. The information processing apparatus according to claim 1, wherein the generation unit sets, as the first area, an area having a size or a shape corresponding to information of the handwritten input acquired from the first user apparatus. A recognition unit for recognizing the character when the first handwritten image is a character;
The information processing apparatus according to claim 2, wherein when the first handwritten image is recognized as a character, the generating unit uses the size of the character or the direction in which the character is arranged as information of the handwritten input. A storage unit for storing a history of acquired handwritten input information;
4. The generation unit according to claim 1, wherein an area having a size or a shape corresponding to a handwritten image estimated to be input from the history accumulated for the first user device is used as the first area. The information processing apparatus according to item 1. When the first handwritten image does not fit in the first area, the generation unit enlarges the first area to a size that fits the first handwritten image, and enlarges the second area. The information processing apparatus according to claim 1, wherein the information processing apparatus is moved to a position that does not overlap with the information processing apparatus. When the input of at least one of the first handwritten image and the second handwritten image is completed and a gap is generated between both images, the generating unit includes the first handwritten image and the position at which the gap is filled. The information processing apparatus according to any one of claims 1 to 5, wherein one or both of the second handwritten images are moved. An image acquisition unit for acquiring a designated image designated by a user from the first user device;
The information processing apparatus according to any one of claims 1 to 6, wherein the generation unit generates the screen data representing the acquired designated image arranged in the first region.

Images