JP2004246454A - Image forming program, and image forming device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To display information of a footnote appended to an area of one portion within an image to easily understand correspondence to an assigned area within the image. <P>SOLUTION: When preparing an output screen 60 including an image column 61 for displaying the whole image 61 of which the one portion is assigned as an assigned area 611, and including an annotation column 62 for displaying annotation information 62b related to the assigned area, in one screen, a display format such as a line color for a frame of the assigned area 611 displayed in the image column 61 is made to differ in response to attribute of the annotation information 62b. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an image forming program and an image forming apparatus for forming an image by a computer, and more particularly to an image forming program and an image forming apparatus for creating an image including a comment given by a viewer of an image on the Internet or the like.
[0002]
[Prior art]
In recent years, with the spread of digital cameras, the use of image databases for organizing captured images has been expanding. These image databases can be searched and browsed by a program installed in a computer. When arranging images, a comment corresponding to each image is input, and each image and its corresponding comment can be displayed on a display screen at the same time and browsed.
[0003]
In addition, there are many sites (photo sites) in which image data is registered on a server accessible to the Internet so that users can browse images through the Internet. In this case, a program for searching for and browsing the images is provided on the server, and each user can operate the image database by accessing the photo site.
[0004]
In these photo sites, an unspecified person can browse registered images and transmit a comment on each image. Then, comments and images can be displayed and browsed simultaneously by unspecified persons in association with each other.
[0005]
For example, in Japanese Patent Application Laid-Open No. 2002-55748, when annotating an image, a specific area of the image is determined, and metadata associated with the icon is associated with the area. Discloses a technique for annotating a part of the entire image.
[0006]
Also, for example, when the registrant of an image wants to add a comment to the photo site, it is more convenient to use audio data or the like to add a comment to an image taken by a digital camera. That is, most digital cameras have only a cross key and a few buttons as input devices, and it is extremely troublesome to add a comment in a text format. Therefore, some digital cameras have an after-recording function that allows a comment to be input by voice to a captured image.
[0007]
When an image photographed by a digital camera having the post-recording function is registered in the photo site, a comment input by the post-recording can be used as audio data in the photo site, thereby enabling text data to be registered. Can be handled in the same way as data.
[0008]
When a comment is added to an image using multimedia data such as audio data and video data in addition to text data, multimedia data such as audio data and video data are displayed and played on the screen simultaneously, unlike text data. It is necessary to handle them differently. However, it is inconvenient for viewers as to which region of the image the comment added is multimedia data, unless it is necessary to actually reproduce the associated comment.
[0009]
[Patent Document 1]
JP, 2002-55748, A
[Problems to be solved by the invention]
Therefore, a technical problem to be solved by the present invention is to provide an image forming program and an image forming method capable of displaying information of an annotation added to a partial area of an image in an easy-to-understand manner in correspondence with a designated area in the image. It is to provide a device.
[0011]
[Means for Solving the Problems]
In order to solve the above-described problem, the present invention provides a single-screen image field for displaying an entire image in which a partial area is designated as a designated area, and an annotation field for displaying annotation information associated with the designated area. A computer for forming the output screen information included in the annotation information by associating the annotation information having various attributes with the designated area according to the attribute of the annotation information associated by the annotation adding means. And an image forming program for functioning as an image forming means for forming the output screen information by changing a display format of the designated area displayed in the image column.
[0012]
In the above configuration, the image forming program causes the computer to display, on a single screen, an image column for displaying an entire image specified with a partial area as a specified area, and an annotation field for displaying annotation information associated with the specified area. Is a program for forming output screen information included in the program. The annotation adding means can use annotation information having different attributes, and can associate the annotation information with the specified area specified on the image. As the attribute of the annotation information, specifically, a file type such as text data or multimedia data, a position where an annotation is added such as a comment hierarchy, or a combination thereof is given.
[0013]
The image forming means forms the output screen information by changing the display format of the designated area according to the attribute of the annotation information. Examples of the display format of the designated area include a color of a frame line surrounding the area, a player, a pattern, a thickness, a discoloration of an image in the designated area, a fill in the designated area, a discoloration, and a pattern.
[0014]
According to the above configuration, since the output format of the designated area is different depending on the attribute of the associated annotation information, the viewer who looks at the output screen determines what attribute the injection cavity information is attached to. It can be easier to judge. In this case, by making the display characteristics of the annotation information and the display characteristics of the designated area common, it is possible for the viewer who has viewed the output screen to more easily understand the correspondence.
[0015]
In the above configuration, it is not necessary to display the designated area associated with all the annotation information added to the image. For example, only the designated area associated with the annotation information displayed in the annotation column is displayed. You may make it display on an image.
[0016]
The image creation program of the present invention can be specifically configured in various modes as described below.
[0017]
Further, on the output screen information, the computer is caused to function as specifying means for specifying either the annotation information or the specified area, and when the image forming means is specified by the specifying means, the specified It is preferable to form the output screen information by changing the display format of both the information and the information associated with the specified information.
[0018]
In the above configuration, the display format of both the specified information and the information associated therewith is changed by specifying the annotation information or the specified area on the image by the specifying unit. Examples of the display format change include, for example, a change in the color and line type of the frame line for the designated area as described above, and examples of the annotation information include a change in the display color and reproduction in the case of multimedia data. Can be illustrated.
[0019]
According to the above configuration, it is possible to clarify the correspondence between the specified designated area and the annotation information, and it is possible to clarify the information (the commentary) associated with the information (the designated area or the annotation information) specified by the specifying unit. (Information or designated area) can be displayed in an easy-to-understand manner.
[0020]
Further, in the above configuration, for example, when one area associated with a plurality of pieces of annotation information is specified, for example, annotation information for changing the display format is changed by an operation of a viewer or the like, They can be played in order.
[0021]
Preferably, the computer further functions as importance determination means for determining importance information for each of the annotation information, the image forming means, based on the importance information determined by the importance determination means, The output screen information in which the display format of the annotation information is changed is formed.
[0022]
In the above configuration, the importance determining unit determines the importance information for each piece of annotation information based on information such as the number of times of browsing, the length of the reproduction time, and the like. Then, based on the determined importance information, output screen information is formed by changing the display format of the annotation information. The display format can be changed, for example, by discoloring the annotation information, displaying the annotation information in a different size, changing the design, displaying a diagram indicating the importance or text information near the annotation information, and the like. it can.
[0023]
Further, the present invention provides an output screen information including, on one screen, an image field for displaying an entire image specified with a partial area as a specified area, and an annotation field for displaying annotation information associated with the specified area. An image forming apparatus for forming an image, comprising: an annotation adding unit that associates the annotation information associated with the designated area with the attribute according to the attribute; and an attribute of the annotation information associated by the annotation adding unit. And an image forming unit that forms the output screen information by changing a display format of the designated area displayed in the image field according to the image forming apparatus.
[0024]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, a computer system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a configuration diagram of a computer system according to the present embodiment. The computer system according to the present embodiment is a network system connected to the Internet 4. The computer system includes a web server 1 functioning as an image creation device of the present invention, and a plurality of user terminals 2 and 3 connectable to the server 1 via the Internet 4. The user terminals 2 and 3 can operate the server 1 by accessing a site established on the web server 1 through the Internet.
[0025]
FIG. 2 is a block diagram showing a configuration of a web server of the computer system according to FIG. The web server 1 can be used by connecting to a mouse 5 and output devices 6 such as a display and a printer as external devices. When connected to the external device, the external device interface 11 controls. Also, an I / O interface 9 for controlling the connection with the Internet 4, a processor 7 for controlling the entire processing of the web server 1, a memory 10 functioning as a calculation area of the processor, and a storage device 8 for storing an image creation program and a database. And are connected to each other by a bus 12.
[0026]
The storage device 8 includes a plurality of programs and databases described below, and when these are activated as appropriate, causes the processor 7 of the web server to execute various functions described below. The system program 8a is an image creation program for executing main processing and calculations when the web server 1 functions as an image creation device. The user database is a database for managing users who can operate the system program 8a. The image database 8c is a database for managing images to be arranged on a web site and metadata as annotations added to the images. The web data 8d is web data used for a site allocated to the web server 1.
[0027]
Next, a processing flow of the computer system according to the present embodiment will be described. FIG. 3 is a flowchart of an operation process in the computer system. First, the user operates his / her user terminals 2 and 3 to access a site established on the web server via the Internet 4 using an arbitrary web browser (# 10). Thereby, the program stored in the server 1 is started, and the login screen 20 as shown in FIG. 4 is displayed on the display screen of the user terminal.
[0028]
If the user has already acquired the user ID, the user can log in by selecting the GO button 23 after inputting the user ID and password in the text boxes 21 and 22, respectively (# 12). Thereby, the process proceeds to step # 14, and the folder selection process shown in FIG. 6 is called. When the end button 25 is selected on the login screen 20, the program ends.
[0029]
On the other hand, if the user has not yet acquired the user ID, the user proceeds to step # 13 by selecting the user registration button 24, and the user registration screen 30 shown in FIG. 5 is displayed on the display screen. The user inputs the first name, last name, and e-mail address in the text boxes 31, 32, and 33 on the user registration screen 30.
[0030]
Also, the user inputs the desired user ID and password into the text boxes 34 and 35, respectively. Thereafter, the password is again entered in the text box 36 for confirmation. When the user registration button 37 is selected after these inputs are completed, the information is transmitted to the web server 1, and the web server 1 inquires the user database as to whether the input user ID has been acquired. If the input user ID is not used in the user database, the user can log in, and the process proceeds to step # 14 to call the folder selection process shown in FIG.
[0031]
On the other hand, if the same user ID has already been acquired, a warning to that effect is made, the user ID text box 34 is blanked, and the user registration screen is displayed again to prompt the user to input a new user ID.
[0032]
Note that if the end button 38 is selected at an arbitrary timing on the user registration screen 30, the program ends.
[0033]
In the folder selection process, a folder selection screen 40 as shown in FIG. 7 is displayed. A list box 42 is provided in the folder selection screen 40. In the list box 42, a list of names of folders in which the image files are stored is displayed, and the user operates the scroll bar 41 to scroll and browse the folders as necessary.
[0034]
FIG. 6 is a flowchart showing the flow of processing on the folder selection screen. On the folder selection screen, in step # 16, it is monitored whether an event has occurred due to a user operation. When an event occurs, a process corresponding to each event is performed. The user operates the scroll bar 41 as necessary on the screen as described above, scrolls the display of the list box 42, and selects an arbitrary folder. Thereafter, when the selection button 43 is selected, the process proceeds to step # 17, and the process proceeds to the image list process, and the image list screen is displayed.
[0035]
When the return button is selected on the folder selection screen 40, the screen returns to the login screen 20 in step # 18. When the end button 45 is selected, the program ends.
[0036]
FIG. 8 is a screen display example of the image list screen. The image list screen 50 displays a list of image files stored in the folder selected in the previous step. A list box 51 is provided in the image list screen 50. In the list box 51, the images of the image files stored in the image database 8c in the web server 1 are displayed as thumbnails. By operating the scroll bar 58 as needed, the user can scroll the image displayed in the list box 51 and browse the image.
[0037]
FIG. 9 is a flowchart showing the flow of processing on the image list screen. On the image list screen, in step # 171, it is monitored whether or not an event has occurred due to a user operation. When an event occurs, a process corresponding to each event is performed.
[0038]
When the upload button 53 is selected, the process proceeds to step # 172, where a file designation screen 531 shown in FIG. 10 is displayed. The file designation screen 531 can designate an image file registered in the image database 8c of the web server 1.
[0039]
The file designation screen 531 is provided with a plurality of text boxes 532 for inputting file names of image files to be uploaded to the web server 1. The user inputs the file names of the image files stored in the local folders of the user terminals 2 and 3 operated by the user into the text box 532 (# 172). At this time, by operating the reference button 533, an image file can be searched from the local folder of the user terminal, and the file name can be input to the text box 532.
[0040]
When the enter button 535 is selected after inputting the file name in the text box 532, the designated file is uploaded in step # 173, the image file is added in the folder selected on the folder selection screen 40, and the image file is added in step # 181. The image list screen 50 shown in FIG. 8 is displayed in a state after the change in which is added. On the other hand, if the return button 534 is selected in steps # 172 and # 173, the upload is not performed and the image list screen 50 is displayed via step # 181. When the end button 536 is selected, the program ends.
[0041]
When the delete button 54 is selected from the event monitoring state (# 171) on the image list screen 50, the image file is deleted in step # 175. Specifically, a check box 59 a and a file name 59 b are added to the image 58 displayed in the list box 51, and the image file whose check box 59 a has been checked in advance is selected by selecting the delete button 54. Removed from. Then, the image list screen 50 is displayed in a state after the change in which the image file has been deleted (# 181).
[0042]
When the combo box 52 for changing the number of displayed images is operated from the event monitoring state (# 171) on the image list screen 50, the process proceeds to step # 176. The combo box specifies the number of image files that can be displayed at one time in the list box 51 of one screen. In step # 176, the number of image files selected by the combo box 52 is displayed in the list box 51. Is done. Then, the image list screen 50 is displayed in a state after the number of displayed image files has been changed (# 181). If the number of image files to be displayed is large and not all of them can be displayed in one list box 51, the scroll bar 58 is operated to scroll and display the images in the list box 51. Can be.
[0043]
When the previous page link 56a or the next page link 56b is selected from the event monitoring state (# 171) on the image list screen 50, the process proceeds to step # 177 and step # 178, respectively, and switches to the previous page or next page screen. The image list screen 50 is displayed in a state after the image file has been changed to the image data displayed on the previous page or the next page (# 181).
[0044]
When an operation such as clicking or double-clicking on an image in the list box 51 is performed from the event monitoring state (# 171) on the image list screen 50, the process proceeds to step # 179, where the comment bulletin board process is called and shown in FIG. The comment bulletin board screen is displayed.
[0045]
When the return button 55 is selected from the event monitoring state (# 171) on the image list screen 50, the folder selection process of step # 14 is called, and the folder selection screen shown in FIG. 7 is displayed.
[0046]
When the end button 57 is selected at an arbitrary timing on the image list screen 50, the program ends.
[0047]
FIG. 11 shows a display example of the comment bulletin board screen. The comment bulletin board screen 60 has an image 61 of the image file specified on the image list screen 50, a text box 64 for inputting a comment, a list box 62 for displaying a list of comments published on the bulletin board, and a comment attached by a video image. A playback screen 71 for playing back the video when the video is played, a video toolbar 72 for performing processing such as playback and recording of comments using video data, and an audio toolbar 68 for performing processing such as playback and recording of comments using audio data. In addition, an interest level display button 75 for displaying the interest level totalized based on the comment attached to the image, a combo box 76 for selecting a totaling method when the interest level display is performed, and the like are displayed.
[0048]
12, on the comment bulletin board screen 60, as shown in FIG. 12, a comment creation process (including a process of deleting a registered comment) in step # 21 and a comment browsing process (display of interest level of image) in step # 21. 2) is performed. First, the comment creation process (# 21) will be described.
[0049]
FIG. 13 is a flowchart showing the flow of the initial processing of the comment creation processing. In the comment creation processing, whether or not an event has occurred by a user operation is monitored on the comment bulletin board screen (# 23). When an event occurs, the following processing is performed.
[0050]
FIG. 14A is a diagram showing an image displayed on the comment bulletin board screen 60. The image 61 is composed of one piece of image data, and is managed in a state where the entire area of the image is divided into small cells 615 as shown in FIG. That is, it is uniformly determined to which cell an arbitrary point on the image belongs. Since the cell is a conceptual one for managing the area of the image 61, the image divided into the cells shown in FIG. 14B is not displayed on the comment bulletin board screen 60. In addition, the size of the cell can be appropriately set according to, for example, the size (the number of pixels) of the image. For example, the pixel itself constituting the image as the minimum unit may be a cell, or every several pixels The area may be managed as one cell.
[0051]
When the user operates a pointing device such as a mouse and moves the pointer 610 on the comment bulletin board screen 60 and drags it on the image 61 (# 24), the start point and the end point of the drag are set to diagonal points (in FIG. A rectangular mark 610 of an input candidate as a point P1 and a lower right point P2) is created and displayed on the image. The rectangular mark displayed at this time blinks. If an unconfirmed rectangular mark (not shown) is already displayed on the image 61, the dragging is newly started on the image, so that the previously displayed rectangular mark is deleted and the current drag is performed. The created rectangular mark is created and displayed.
[0052]
As shown in FIG. 14B, the rectangular mark is uniformly determined by the coordinates (x1, y1) and (x2, y2) of the diagonal points P1 and P2 on the image. Then, a cell 615x existing in an area common to the rectangular mark 611x is extracted, and a cell corresponding to the rectangular mark 611 is determined.
[0053]
In this way, when a selection area is designated (# 25), a comment can be associated with each desired selection area. As a process for inputting the comment, a metadata input process (# 26) is performed. Note that, as metadata in the present embodiment, three types of text data, audio data, and video data can be used, and each is input as follows.
[0054]
FIG. 15 is a flowchart showing the flow of the metadata input process. In the metadata input processing (# 26), the processing and buttons to be operated on the screen differ depending on the type of input metadata.
[0055]
First, a case where text data is used as input metadata will be described. In inputting text data (# 126), the user inputs text data to be input into the text box 64 (# 127). At this time, if an input error or re-input is made, the text data in the text box 64 is deleted by selecting the clear button 67 (# 128) (# 130). Enter again.
[0056]
When the input of the text data in the text box 64 is completed and the text data is to be registered as a comment, the user selects the add button 65 (# 129). By selecting the add button 65, a process for registration described later is started.
[0057]
Next, a case where audio data is used as input metadata will be described. When the user selects the recording button 68a of the voice toolbar 68 in the input of the voice data (# 226), the user terminal drives a local sound input device to start recording. The maximum recording time of the audio data is determined in advance, and the remaining recordable time can be confirmed by the indicator 68d of the audio toolbar 68. When the indicator 68d moves to the right end in FIG. 11, the recording is automatically terminated due to the lapse of the maximum recordable time, but the recording is stopped halfway by the user selecting the stop button 68c at an arbitrary timing. Can be stopped. The recorded audio data is stored in a temporary folder of the user terminal, and an icon 68e indicating that recorded audio data exists is displayed near the audio toolbar 68.
[0058]
When the recorded audio data is stored in the temporary folder of the user terminal and the audio data is to be deleted, the clear button 70 is selected (# 228) to delete the audio data. At the same time, the icon 68e is deleted from the screen (# 230).
[0059]
When the user wants to check the recorded voice data, he or she selects the play button 68b of the voice toolbar 68. Reproduction of audio data can be performed from an arbitrary position by moving the slide of the indicator 68d to an arbitrary position and then selecting the reproduction button 68b. During the reproduction of the audio data, the reproduction can be stopped by selecting the stop button 68c.
[0060]
When registering the recorded audio data, the user selects the add button 69 (# 229). When the add button 69 is selected, a registration process described later is started.
[0061]
Next, a case where video data is used as metadata will be described. In the input of video data (# 326), when the user selects the recording button 72a of the video toolbar 72, the user terminal drives a local video input device to start recording. The maximum recording time of the video data is determined in advance, and the remaining recordable time can be confirmed by the indicator 72d of the video toolbar 72. When the indicator 72d moves to the right end in FIG. 11, the recording is automatically terminated due to the elapse of the maximum recordable time, but the user selects the stop button 72c at an arbitrary timing to stop the recording in the middle. Can be stopped. The recorded video data is stored in a temporary folder of the user terminal, and the first image is displayed on the playback screen 71.
[0062]
When the recorded video data is stored in the temporary folder of the user terminal and the video data is to be deleted, the user selects the clear button 74 (# 328) to delete the video data. At the same time, the first image displayed on the reproduction screen 71 is deleted (# 330).
[0063]
If the user wants to check the recorded video data, the user selects the play button 72b on the video toolbar 72. The video data can be reproduced from an arbitrary position by moving the slide of the indicator 72d to an arbitrary position and then selecting the reproduction button 72b. During the reproduction of the video data, the reproduction can be stopped by selecting the stop button 72c.
[0064]
When the user wants to register the recorded video data, the user selects the add button 73 (# 329). By selecting the add button 73, a process for registration described later is started.
[0065]
As described above, when the input of the metadata to be registered ends according to the type, and the additional buttons 65, 69, 73 for registration corresponding to the respective types of metadata are selected (# 129, # 229). , # 329), the designated area determined in step # 25 above is acquired (# 31), registered in the image database of the web server 1, and displayed as a comment related to the designated area in the image 61 on the comment bulletin board screen. Added to the bulletin board.
[0066]
The display position of the comment in the list box 62 is determined by whether or not the check box 63 in the list box 62 is checked. Specifically, when the check box 63 for any registered comment is checked, the comment is added below the checked comment (# 33). Only one check box 63 can be checked. on the other hand. If the check box 63 is not checked, the registered comment is added as the top comment (# 34). At this time, the area indicated by the comment and the input metadata are stored in the image database 8c in association with each other. As described above, the coordinate values of the two diagonal points on the image are used as the information of the designated area.
[0067]
The metadata registered as a comment in the list box 62 has a different display form in the list box 62 depending on its type. When the comment is text data, the character information 62b is directly displayed. If the comment is audio data, the audio icon 62c is displayed at the position determined in steps # 33 and # 34. If the comment is video data, a video icon 62d is displayed at the position determined in steps # 33 and # 34.
[0068]
Next, a process of deleting a registered comment will be described. FIG. 16 is a processing flowchart of comment deletion. The comment may be deleted so that only the comment registered by the user can be deleted, or an arbitrary person may be able to delete any comment.
[0069]
When the comment bulletin board screen is displayed on the screen, it is in the event monitoring state as described above. In this state, when deleting a registered comment, first, a comment to be deleted is selected (# 40). Specifically, the user operates the mouse or the like to check the check box 63 of the comment to be deleted. When the comment to be deleted is determined, the selected comment is deleted by selecting a delete button 77 (# 41). The process for deleting a comment depends on the layer in which the selected comment is provided. That is, when the comment to be deleted is the comment of the lowest layer, the comment is deleted as it is by selecting the delete button 77 (# 43). On the other hand, if the comment is not the lowest comment, it is confirmed whether to delete the comment together with the comment existing in the lower layer than the selected comment (# 44). Delete with
[0070]
Next, the comment browsing process (including the process of displaying the degree of interest of an image) in step # 22 on the comment bulletin board screen 60 will be described. 18 and 19 are flowcharts showing the flow of the comment browsing process.
[0071]
According to the size of the list box 62, the number of comments that can be displayed on one screen is limited. Comments that cannot be displayed in the list box 62 can be switched and displayed by scrolling by operating the scroll bar 62a of the list box 62. In the image 61, only the designated area frame 611 associated with the comment displayed in the list box 62 is displayed, and the comment displayed in the list box 62 is switched according to the operation of the scroll bar 62a. As a result, the display of the designated area frame 611 is also switched and displayed. The designated area frame 611 has a different color depending on the type of the associated metadata. For example, text data is displayed in black, audio data is displayed in yellow, and video data is displayed in blue.
[0072]
The operation of browsing the registered comments includes a method of browsing by operating a comment or an icon displayed in the list box 62, and a method of selecting and displaying an area of the image 61. In any case, the comment in the list box 62 and the designated area of the image 61 are linked, and when any one is selected, the other corresponding one is highlighted.
[0073]
First, the browsing process when a comment in the list box 62 is selected will be described.
[0074]
When the mouse pointer 610 (refer to FIG. 14) does not exist in either the image 61 or the list box 62, it is in a state of waiting for an event to occur, and the color of the comment 62b, the icons 62c, 62d, and the border line of the related designated area Is in the default state (# 51), comment characters are displayed in black, icons are displayed in standard display color (black), and the frame of the designated area is displayed in a color corresponding to the type of metadata as described above. ing.
[0075]
In browsing the text comment, when the mouse pointer moves over the text comment in the list box, the object related to the comment (the text comment and the designated area frame of the comment) turns red (# 153). When the mouse is clicked without moving the mouse pointer out of the text comment (No in # 154) (# 155), the related area is extracted and enlarged and displayed on the reproduction screen 71 (# 156). When the mouse is clicked again (# 157), the reproduction screen 71 returns to the whole image (# 158), and the browsing of the text comment ends.
[0076]
On the other hand, if the mouse pointer deviates from the text comment at any timing of these processes, the color of the text comment and the designated area frame returns to the original black, the browsing process ends, and a state of waiting for an event is set.
[0077]
The reading of the voice comment can be performed by moving the mouse pointer over the voice icon.
[0078]
The voice comment icon 62c in the list box 62 is determined by determining the importance of the comment based on the average of the percentage of time the voice comment was played, and changing the design of the icon in the list box 62 according to the importance. indicate. Specifically, the importance is determined based on the average of (time actually reproduced by the user image) / (recording time of the comment), and the importance is determined to be 0 to less than 33%, 33 to less than 66%, 66 to 100%. 17 and three types of icons are displayed as shown in FIG. From the icon design, the user can determine the importance of the audio data in advance without actually reproducing the audio data, and can use the icon as an index from which audio comment should be reproduced.
[0079]
When the mouse pointer moves over the voice icon, the voice icon as a related object and the designated area frame of the comment turn red (# 253). Thereafter, reproduction of the audio data is started (# 254). When the mouse is clicked without moving the mouse pointer out of the voice icon 62c (No in # 255) (# 256), the related area is extracted and enlarged and displayed on the reproduction screen 71 (# 257). When the mouse is clicked again (# 258), the reproduction screen 71 returns to the whole image (# 259), and the browsing of the voice comment ends.
[0080]
On the other hand, if the mouse pointer deviates from the audio icon at any timing of these processes, the reproduction of the audio data ends, and the color of the audio icon and the designated area frame returns to the original color, and the browsing process ends. And enters the event waiting state.
[0081]
Viewing a video comment can be done by moving the mouse pointer over the video icon.
[0082]
Similar to the audio icon, the video comment icon in the list box 62 determines the importance of the comment by the average of the percentage of time the video comment was played, and the icon in the list box 62 according to the importance. Is displayed with a different design. Therefore, the user can determine the importance of the video data in advance from the icon design without actually reproducing the video data, and can use the video data as an index from which video comment should be reproduced.
[0083]
When the mouse pointer moves over the video icon, the video icon as a related object and the designated area frame of the comment turn red (# 353). Thereafter, the reproduction of the video data is started (# 354). When the mouse pointer is clicked without moving the mouse pointer out of the video icon 62c (No in # 355) (# 356), the related area is extracted and enlarged and displayed on the reproduction screen 71 (# 357). When the mouse is clicked again (# 358), the reproduction screen 71 returns to the whole image (# 359), and the browsing of the text comment ends.
[0084]
On the other hand, if the mouse pointer deviates from the video icon at any timing of these processes, the reproduction of the video data ends, the color of the video icon and the designated area frame returns to the original color, and the browsing process ends. And enters the event waiting state.
[0085]
Next, the browsing process when the designated area of the image 61 is selected will be described. When the designated area of the image is selected, comments are browsed when a plurality of comments (including hierarchically attached comments) are attached to one designated area, or when a point on the image is assigned to a plurality of points. The processing procedure differs from the case where the comment in the list box 62 is selected, as described below, because it may overlap the designated area.
[0086]
As described above, in the comment bulletin board screen, when an event occurrence is waiting, the color of the comment 62b, the icons 62c, 62d, and the frame color of the related designated area is in a default state (# 51). ). In this state, when the mouse pointer moves up to the image 61, it is determined whether or not the position is a designated area with a comment (# 70). If it is not the designated area, the process for browsing is not performed, and the event standby state is continued.
[0087]
If the point where the mouse pointer is moved is the designated area, it is determined whether or not a plurality of designated areas exist at the point (# 71). If it is not a plurality of designated areas (No in # 71), it is determined whether the number of comments associated with the designated area is one or more (# 72). When the number of comments associated with the designated area is singular (Yes in # 72), the following processing is performed according to the type of comment.
[0088]
If the comment associated with the designated area is a text comment, the same processing A as that for reading the text comment is performed (see steps # 153 to # 158 in FIG. 18). That is, the text comment and the designated area frame associated with the designated area turn red, and the area is enlarged and displayed by clicking the area with the mouse. When the mouse pointer leaves the area, the text comment in the designated area frame and the list box 62 changes to the original black.
[0089]
If the comment associated with the designated area is a voice comment, the same processing B as that for browsing the voice comment is performed (see steps # 253 to # 259 in FIG. 18). That is, the audio icon and the specified area frame associated with the specified area turn red, the audio data is reproduced, and the area is enlarged and displayed by clicking the area with the mouse. When the mouse pointer moves out of the area, the reproduction of the audio data ends, and the audio icon in the designated area frame and the list box 62 changes to the original color.
[0090]
If the comment associated with the designated area is a video comment, the same processing C as that for browsing the video comment is performed (see steps # 353 to # 359 in FIG. 18). That is, the video icon associated with the designated area and the designated area frame turn red, the video data is played while the mouse pointer is in the area, and the area is clicked with the mouse to change the area. Enlarged display. When the mouse pointer moves out of the area, the reproduction of the video data ends, and the video icon in the designated area frame and the list box 62 changes to the original color.
[0091]
When there are a plurality of comments associated with the designated area (No in # 72), it is determined whether all of the plurality of comments are text comments (# 74), and the respective processes are performed as follows. .
[0092]
When all of the plurality of comments associated with the designated area are text comments (Yes in # 74), the same processing A as that for browsing the text comments is performed (see steps # 153 to # 158 in FIG. 18). That is, all the text comments and the specified area frame associated with the specified area turn red, and the area is enlarged and displayed by clicking the area with the mouse. When the mouse pointer moves out of the area, all the text comments in the designated area frame and the list box 62 are changed to the original black.
[0093]
If all of the plurality of comments associated with the designated area are not text comments (No in # 74), that is, if there is a text comment and an audio comment or a video comment, or if there is only an audio comment or a video comment, the related All comments (text comments, audio icons, and video icons) associated with the specified area as the object and the specified area frame turn red (# 75). Then, among the associated comments, the audio data and the video data are sequentially played back in ascending order of the date and time when the comment was added (# 76).
[0094]
If it is desired to stop the reproduction of the comment during the reproduction of the audio data and the video data, the mouse is double-clicked (# 78), and the reproduction of the next comment is started (# 79). When the mouse is clicked during the reproduction of the audio data and the video data, the image of the related area is extracted and enlarged and displayed on the reproduction screen 71 (# 81). When the mouse is clicked again (# 82), the reproduction screen 71 returns to the whole image (# 83), and the browsing of the comment ends.
[0095]
On the other hand, if the mouse pointer goes out of the designated area at any timing of these processes, the reproduction of the audio data and the video data ends, and the comment in the designated area frame and the list box 62 returns to the original color. After that, the browsing process is completed, and an event waiting state is set.
[0096]
Next, a case where a plurality of designated areas exist at the point where the mouse pointer is moved (Yes in # 71) will be described. In this case, it is determined whether or not all the comments associated with all the specified area frames included in the point where the mouse pointer is present are text comments (# 85), and the respective processes are performed as follows.
[0097]
When all the comments associated with the plurality of designated areas existing at the point where the mouse pointer is moved are all text comments (Yes in # 85), the same processing A as the above-described browsing of the text comments is performed. (See steps # 153 to # 158 in FIG. 18). That is, all the text comments associated with the specified area and all the specified area frames existing at the point where the mouse pointer is moved are changed to red.
[0098]
By clicking the mouse at this position, the area is enlarged and displayed. At this time, the range of the image enlarged and displayed is the image in the bounding box 612 as shown in FIG. The bounding box 612 is a rectangular area circumscribing a plurality of designated areas. That is, when three designated areas 611a to 611c exist at the position where the mouse pointer 610 exists, a rectangular area 612 circumscribing the three designated areas becomes a bounding box. When the mouse pointer moves and deviates from any of the designated areas, all the text comments in the designated area frame and the list box 62 change to the original color. That is, when the mouse pointer departs from the overlapping position of all the specified areas, all the specified area frames and comments return to the original colors.
[0099]
If all of the plurality of comments associated with the designated area are not text comments (No in # 85), that is, if there is a text comment and an audio comment or a video comment, or if there is only an audio comment or a video comment, (See steps # 75 to # 84 in FIG. 19). In other words, all the comments (text comments, audio icons, and video icons) associated with the specified area, which is a related object, and the specified area frame turn red. Then, among the associated comments, the audio data and the video data are sequentially reproduced in ascending order of the date and time when the comment was added.
[0100]
If it is desired to stop the reproduction of the comment during the reproduction of the audio data and the video data, the mouse is double-clicked as it is, and the reproduction of the next comment is started. When the mouse is clicked during the reproduction of the audio data and the video data, the image of the related area is extracted and enlarged and displayed on the reproduction screen 71. The range of the image to be enlarged and displayed is the range of the bounding box 612 as described above. When the mouse is clicked again, the reproduction screen 71 returns to the whole image, and the browsing of the comment ends.
[0101]
On the other hand, if the mouse pointer goes out of the designated area at any timing of these processes, the reproduction of the audio data and the video data ends, and the comment in the designated area frame and the list box 62 returns to the original color. After that, the browsing process is completed, and an event waiting state is set.
[0102]
Next, a case where the interest level display button 75 is selected on the comment bulletin board screen 60 will be described. When the interest level display button 75 is selected, the server counts the interest levels (points) for each cell 615 (see FIG. 14) according to a predetermined evaluation criterion, and displays the result in the form of a graph in an interest level distribution screen. (See FIG. 21) is displayed on the user terminals 2 and 3. Specifically, the following processing is performed.
[0103]
First, when displaying the interest level distribution screen 70, the user selects an interest level evaluation criterion from the combo box 76. Examples of the degree of interest evaluation criterion include the number of registered comments, the number of times of browsing, the total of browsing time, and the like. When one cell belongs to a plurality of designated areas or a plurality of comments are attached to the same designated area, the degree of interest (point) given by the comment attached to the designated area is accumulated. It can also be summed and counted.
[0104]
In addition, the comment data is configured to be classified and totaled for each type. For example, the degree of interest in the audio comment and the video comment may be evaluated using a ratio of the time actually reproduced by the user to the total reproduction time of the audio data and the video data. Further, a plurality of items may be combined and evaluated, and for example, the degree of interest may be calculated by performing some kind of calculation such as “number of browsing times + rank of browsing time × 3”.
[0105]
When the interest level evaluation criterion is determined by operating the combo box 76, the web server 1 acquires this information. (# 61). Next, data is tabulated for each cell in accordance with the criterion, and a distribution chart is created using the point data of each tabulated cell (# 62). When the creation of the distribution chart is completed, the information is output to the user terminal, and the user terminal receiving the information opens another window to display the interest distribution screen 70 (# 63).
[0106]
As the display screen, as shown in FIG. 21, an area graph 61G or the like corresponding to the point of the degree of interest for each mesh is preferably used. That is, graphing is performed so that the height of each cell 615 differs according to the point of each cell. In addition, as an example of the graph, in addition to the graph illustrated in FIG. 21, for example, the image 61 itself may be embossed. Alternatively, the distribution may be displayed based on information for each type of comment, such as only text information or only audio.
[0107]
When the close button 74 or the end button 73 is selected on the interest distribution screen 70, the interest distribution screen is closed, the screen returns to the comment bulletin board screen 60, and an event standby state is set.
[0108]
As described above, according to the computer system having the above configuration, an unspecified number of persons can comment on an image posted on a site using text data and multimedia data for a partial area of the photograph. Can be. In addition, since the color of the frame indicating the area is displayed differently depending on the type of data, it is possible to immediately determine which part has what kind of comment.
[0109]
Also, when browsing, by selecting one of the designated area frame or the comment with the mouse, the other associated color is changed, so that it is possible to easily identify which comment is for which area. it can. The icon indicating the comment displayed in the list box changes its display according to the importance. Therefore, by looking at the display, the importance of the audio data can be determined in advance without actually reproducing the audio data, and the index can be used as an index of the audio comment to be reproduced.
[0110]
The present invention is not limited to the above embodiment, but can be implemented in various other modes.
[0111]
For example, the present computer system is configured by a network system using the Internet. However, for example, a device having the same function as the web server of the present embodiment may be used alone without using a network. For example, in this case, a user can register a photographic image at a store or the like, and a third party can use the same device to add a comment.
[0112]
When the display of the designated area is changed according to the type of the metadata, the display may be made different by changing not only the color of the designated area frame but also the line type and the thickness of the frame line.
[Brief description of the drawings]
FIG. 1 is a configuration diagram of a computer system according to an embodiment.
FIG. 2 is a block diagram showing a configuration of a web server of the computer system according to FIG. 1;
FIG. 3 is a flowchart of an operation process in the computer system according to the embodiment.
FIG. 4 is a screen display example of a login screen.
FIG. 5 is a screen display example of a user registration screen.
FIG. 6 is a flowchart showing the flow of processing on a folder selection screen.
FIG. 7 is a screen display example of a folder selection screen.
FIG. 8 is a screen display example of an image list screen.
FIG. 9 is a flowchart showing the flow of processing on an image list screen.
FIG. 10 is a screen display example of a file designation screen.
FIG. 11 is a screen display example of a comment bulletin board screen.
FIG. 12 is a flowchart showing a flow of processing on a comment bulletin board screen.
FIG. 13 is a flowchart showing the flow of initial processing of comment creation processing.
14A and 14B are diagrams illustrating images used on a comment bulletin board screen, in which FIG. 14A is a view illustrating an image displayed on a comment bulletin board screen, and FIG. 14B is a view obtained by dividing the image in FIG. It is a figure showing a state.
FIG. 15 is a flowchart showing the flow of metadata input processing.
FIG. 16 is a processing flowchart of comment deletion.
FIG. 17 is a diagram showing audio comment icons displayed in a list box of the comment bulletin board screen.
FIG. 18 is a flowchart showing the flow of a comment browsing process.
FIG. 19 is a flowchart showing the flow of a comment browsing process.
FIG. 20 is an explanatory diagram of a bounding box.
FIG. 21 is a screen display example of a degree of interest distribution screen.
[Explanation of symbols]
1 Web server 2, 3 User terminal 4 Internet 5 Mouse 6 Output device 7 Processor 8 Storage device 20 Login screen 30 User registration screen 40 Folder selection screen 50 Image list screen 60 Comment bulletin board screen 61 Image 61G Interest distribution graph 61x Cell division Image 611 Designated area frame 611x Designated area 615 Cell 70 Interest level distribution screen

Claims (5)

A computer is used to form output screen information including, on a single screen, an image field for displaying an entire image specified as a specified area, and an annotation field for displaying annotation information associated with the specified area. ,
Annotation adding means for associating annotation information having various attributes with the specified designated area,
An image for functioning as image forming means for forming the output screen information by changing the display format of the designated area displayed in the image field according to the attribute of the annotation information associated by the annotation adding means. Shaping program. 2. The image forming program according to claim 1, wherein the attribute of the annotation information to which the annotation adding unit can perform different association includes a type of a file type of the annotation information. Further, the computer, on the output screen information, the function as a specifying means for specifying any one of the annotation information or the specified area,
The image forming unit, when specified by the specifying unit, forms the output screen information by changing a display format of both the specified information and the information associated with the specified information. The image forming program according to claim 1, wherein: Further, causing the computer to function as importance determination means for determining importance information for each of the individual annotation information,
The image forming unit forms the output screen information in which the display format of the annotation information is changed based on the importance information determined by the importance determining unit. The image forming program according to any one of the above. An image forming apparatus for forming output screen information including an image field for displaying an entire image specified with a partial area as a specified area and an annotation field for displaying annotation information associated with the specified area on one screen. So,
Annotation means for associating the annotation information associated with the designated area with a different attribute according to the attribute,
Image forming means for forming the output screen information by changing the display format of the designated area displayed in the image field according to the attribute of the annotation information associated by the annotation adding means. Image forming apparatus.

Images