JP7027750B2 - Image sharing systems, servers, information terminals, and programs - Google Patents

Description

The present invention relates to an image sharing system or the like in an SNS (Social Network Service) service or the like.

In recent years, SNS services have been provided that publish images taken by information terminals such as smartphones via a network and share them among registered users.

This type of service also provides and uses the function of adding text such as explanations and comments to posted images. However, since the addition of such text is performed manually for each image, it takes time to input the text, and there is a possibility that the impression, impression, and presence at the moment when the photograph is taken may be impaired.

On the other hand, Patent Document 1 discloses an electronic album editing device that inputs voice, converts it into text, and adds it to an image. This electronic album editing device, for example, when you enter a voice comment "Congratulations" on an image taken at a wedding ceremony, it is converted into text data on the spot and combined with the image. No input is required.

Japanese Patent No. 4240867

However, since the electronic album editing device of Patent Document 1 described above is premised on printing or recording the created electronic album on a recording medium, it can be applied as it is when sharing an image like an SNS service. It wasn't a thing.

The present invention has been made in view of such a problem, and an object of the present invention is to provide an image sharing system or the like capable of adding text to an image without impairing the impression, impression, and presence at the moment of shooting.

The first invention for solving the above-mentioned problems is an image sharing system having a plurality of information terminals and a server, wherein the information terminals include a camera for taking an image and a microphone for receiving voice input. A display that displays an image display area for displaying an image captured by the camera, a shutter button of the camera, and a microphone button operated when voice input is performed by the microphone in the same screen. A means, a recording means for adding and recording the voice or a text obtained by converting the voice to a captured image as additional information, and a transmission means for transmitting the image and the additional information to the server. The server has a recording means for recording the image and recording the text obtained by converting the voice as additional information of the image, and the image and the text of the transmission source of the image. It is an image sharing system including a transmission means for transmitting to the information terminal different from the information terminal.

According to the present invention, the user can add the voice or the text obtained by converting the voice to the image by inputting the voice to the image taken by the information terminal, and the server can add the text obtained from the voice to the image. It can be provided to other users along with the image. In the present invention, since the input voice is converted into text and added to the image, manual input of the text becomes unnecessary, and the user directly expresses a fresh impression, impression, and presence at the moment of shooting. It can be added to the image and viewed by other users via the network.

It is desirable that the information terminal has a conversion means for converting the voice into text, and the transmission means of the information terminal transmits the image and the text to the server.
By performing text conversion of voice on the information terminal side, it becomes possible for the user to correct an error or the like in the converted text.

The information terminal displays both the image and the text.
In this way, the text is displayed together with the image as a description or impression of the image.

It is desirable that the information terminal creates layout data in which a plurality of the images and texts attached to the images are arranged in a predetermined layout.
This makes it possible to display a plurality of images on one screen like a photo book.

Further, it is desirable that the information terminal can set the same text for a plurality of images by one voice input.
This makes it possible to add text to a plurality of images at once, and for example, the same explanation or the like can be added to a plurality of images taken during a trip.

The second invention is a server in the image sharing system of the first invention .

The third invention is an information terminal connected to a server via a network, and includes a camera that captures an image, a microphone that receives audio input, and an image display area that displays an image captured by the camera. A display means for displaying a display screen including the shutter button of the camera and the microphone button operated when inputting voice by the microphone in the same screen, and converting the voice or the voice into a captured image. A recording means for adding and recording the text obtained as additional information, a transmission means for transmitting the image and the additional information to the server, an image transmitted from a different information terminal to the server, and the image. The information terminal is provided with a receiving means for receiving the text added as additional information from the server.

The fourth invention is a server connected to a plurality of information terminals of the third invention via a network, an image, and a voice or a text obtained by converting voice or voice added as additional information to the image. A receiving means for receiving the above from the information terminal, a recording means for recording the image and recording the text obtained by converting the voice as additional information of the image, and the image and the text. It is a program for functioning as a transmission means for transmitting to the information terminal different from the information terminal from which the image is transmitted.

A fifth invention is an image display that displays an image captured by the camera of an information terminal having a camera for taking an image and a microphone for receiving voice input and connected to a server via a network. A display means for displaying a display screen including an area, a shutter button of the camera, and a microphone button operated when performing voice input by the microphone, and the voice or the voice in a captured image. A recording means for adding and recording the text obtained by conversion as additional information, a transmission means for transmitting the image and the additional information to the server, and an image transmitted from a different information terminal to the server. It is a program for functioning as a receiving means for receiving text added as additional information to the image from the server.

INDUSTRIAL APPLICABILITY According to the present invention, it is possible to provide an image sharing system or the like that can add text to an image without impairing the impression, impression, and presence at the moment of shooting.

The figure which shows the image sharing system 10. The figure which shows the hardware composition of the information terminal 1. The figure which shows the hardware configuration of a server 3. The figure which shows the image data area 51 and the additional information area 52. The flowchart explaining the flow of processing in an image sharing system 10. An example of the display screen of the display unit 13. An example of the display screen of the display unit 13. An example of the display screen of the display unit 13. An example of the display screen of the display unit 13. The flowchart explaining the flow of processing in an image sharing system 10. An example of the display screen of the display unit 13. The figure explaining the record of comment 80. An example of the display screen of the display unit 13. An example of laying out a plurality of images 63 and text 65.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[First Embodiment]
FIG. 1 is a diagram showing a configuration of an image sharing system 10 according to an embodiment of the present invention.

As shown in FIG. 1, the image sharing system 10 is configured by connecting a plurality of information terminals 1 and an SNS server 3 (hereinafter, simply referred to as a server 3) so as to be able to communicate with each other via a network 8. The network 8 may be wired or wireless.

The information terminal 1 is a computer terminal owned by the user, for example, a mobile terminal such as a smartphone or a tablet terminal.

FIG. 2 is a diagram showing a hardware configuration of the information terminal 1. As shown in FIG. 2, the information terminal 1 is configured by connecting a control unit 11, a storage unit 12, a display unit 13, an input unit 14, a communication control unit 15, a camera 16, a microphone 17, a speaker 18, and the like by a bus or the like. Will be done. However, the present invention is not limited to this, and various configurations can be taken as appropriate.

The control unit 11 is composed of a CPU, ROM, RAM, and the like. The CPU calls and executes a program related to the processing of the information terminal 1 stored in the storage medium such as the storage unit 12 and the ROM to the work area on the RAM. ROM is a non-volatile memory that permanently holds programs such as boot programs and BIOS, and data. The RAM is a volatile memory, and includes a work area used by the control unit 11 to perform various processes while temporarily holding programs and data loaded from the storage unit 12, ROM, and the like.

The storage unit 12 is a flash memory or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS, and the like. These programs and data are read out and executed by the control unit 11 as needed.

In the present embodiment, the storage unit 12 stores application programs and other programs and data necessary for causing the information terminal 1 to execute the processing described later. These programs and data can be downloaded from, for example, the server 3.

The display unit 13 is, for example, a liquid crystal display or the like.
The input unit 14 inputs various settings to the information terminal 1, and in the present embodiment, it is, for example, a touch panel provided on the display unit 13.
The communication control unit 15 is a communication interface that mediates communication via a network or the like, and communicates with a server 3 or the like.

The camera 16 includes an optical lens, an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor), an A / D (Analog / Digital) converter, and the like. The camera 16 photoelectrically converts a subject image input via an optical lens by an image pickup device to generate an analog image signal. Then, the analog image signal is converted into digital image data by the A / D conversion unit.

The microphone 17 converts the input voice into digital voice data by the A / D conversion unit. Digital speech data is converted to text as described below.
The speaker 18 outputs digital audio data as audio.

The server 3 executes processing such as recording of images and the like received from the information terminal 1, transmission of images and the like to different information terminals 1, registration and management of user information, and the like as an SNS server.

FIG. 3 is a diagram showing a hardware configuration of the server 3. The server 3 can be realized by, for example, a computer configured by connecting a control unit 31, a storage unit 32, a display unit 33, an input unit 34, a communication control unit 35, and the like by a bus or the like. However, the present invention is not limited to this, and various configurations can be taken as appropriate.

The control unit 31, the storage unit 32, the display unit 33, and the communication control unit 35 are substantially the same as the control unit 11, the storage unit 12, the display unit 13, and the communication control unit 15 described above. Further, the input unit 34 includes an input device such as a keyboard and a mouse.

In the present embodiment, the server 3 includes a user DB (database) 301, an image DB 302, a comment DB 303, and the like in the storage unit 32.

The user DB 301 stores user information, which is information of registered users of the image sharing system 10. The user information includes the registered user's name, attributes (gender, age), e-mail address, user ID, user settings, and the like.

The image DB 302 stores an image or the like transmitted from the information terminal 1. As shown in FIG. 4, in the image DB 302, an image is recorded in the image data area 51, and data such as text, sound, image ID, user ID, and image shooting date and time are added information in the image data area 52. It is recorded in and these are linked.

The comment DB 303 stores a comment posted on the image.

The image sharing system 10 can also be linked with an SNS server 9 of another business operator via the network 8. The SNS server 9 has a control unit, a storage unit, a display unit, an input unit, a communication control unit, and the like similar to the server 3.

(2. Processing in the image sharing system 10)
Next, the flow of processing executed in the image sharing system 10 will be described.

(2-1. Taking and posting images)
FIG. 5 is a flowchart illustrating the flow of processing in the image sharing system 10, and shows the flow from image capture to posting. S11 to S16 in FIG. 5 are processes executed by the control unit 11 of the information terminal 1, and S17 and S18 are processes executed by the control unit 31 of the server 3.

In the present embodiment, the information terminal 1 activates the application in response to the user's operation (S11), and the screen shown in FIG. 6A is displayed on the display unit 13. The screen is provided with an image display area 40 and an operation area in which a shutter button 41, a microphone button 42, a stop button 43, a save button 44, and the like are displayed. In FIG. 6A, the image currently captured by the camera 16 is displayed in the image display area 40.

When the shutter button 41 is selected on the screen, the information terminal 1 takes a picture with the camera 16 (S12). As shown in FIG. 6B, the captured image 63 is displayed in the image display area 40.

When the user selects the microphone button 42 on the screen, the information terminal 1 receives voice input from the microphone 17 and records the sound (S13). Recording ends when the stop button 43 is selected by the user or when a predetermined time limit has elapsed.

During recording, the voice bar 45 is displayed in the operation area as shown in FIG. 6 (b). In the voice bar 45, the total length indicates the time limit, and the position of the marker (circle portion on the bar) indicates the passage of time during recording.

When the recording is completed, the information terminal 1 converts the input voice into text (S14; conversion means). The text is displayed on the screen as shown in FIG. 6 (c). In the example of FIG. 6C, the text 65 is arranged in the image 63, but the arrangement of the text 65 is not particularly limited. In addition, it is desirable that the conversion from voice to text be compatible with multiple languages.

If there are text conversion candidates, for example, as shown in FIG. 7, the text conversion candidate 66 may be arranged on the screen, and when the conversion candidate 66 is selected, it may be displayed as the text 65. .. In the example of FIG. 7, the conversion candidate 66 is arranged in the operation area in the screen, but the arrangement of the conversion candidate 66 is not particularly limited.

After that, the user selects the save button 44 on the screen. When the save button 44 is selected, the information terminal 1 adds text 65 (additional information) to the captured image 63 and records it in the storage unit 12 (S15). The method of adding the text 65 to the image 63 is not particularly limited. The text 65 may be arranged (embedded) in the image 63, or the image 63 and the text 65 may be linked and recorded separately. Further, data such as the voice that is the basis of the text 65 and the shooting date and time of the image 63 are also recorded as additional information in association with the image 63.

By the above procedure, the text 65 is added to the captured image 63 and recorded in the storage unit 12. This image 63 can be displayed on the display unit 13 of the information terminal 1 together with the text 65. 8A and 8B are examples of the display screen at this time. FIG. 8A shows a case where the text 65 is embedded in the image 63, and FIG. 8B shows a case where the image 63 and the text 65 are recorded separately. Is an example of. In the latter, the image 63 and the text 65 are arranged in a predetermined layout, and in the present embodiment, the text 65 is displayed in the vicinity of the image 63. This layout can be set as appropriate. It is also possible to record only the image 63 without inputting the sound. In this case, after displaying only the image 63 in the same manner as in FIG. 8A, the microphone button 42 is selected. Hereinafter, by performing the same processing as in S13 to S15, the text 65 can be added to the image 63 again as additional information and recorded in the storage unit 12.

Further, in the present embodiment, the text 65 is added to the image 63 each time the image is taken, but it is also possible to set the same text 65 to the plurality of images 63 by one voice input. ..

For example, a thumbnail image display button (not shown) is provided on the screens of FIGS. 8A and 8B, and when this button is selected, a plurality of images 63 that have already been captured are thumbnailed in the image display area 40 as shown in FIG. Display a list of images so that you can select the image 63 to which text is added. By selecting the image 63 to which the text is to be added and performing voice input and text conversion in the same manner as before, the same text 65 can be collectively set for a plurality of images 63. By selecting the save button 44, it is possible to add text 65 to each image 63 and record it.

Return to the explanation of the flowchart. The information terminal 1 transmits (posts) the image 63 and the text 65 added as additional information to the server 3 according to the user's operation (S16; transmission means). As additional information of the image 63, in addition to the text 65, data such as the voice that is the basis of the text 65, the shooting date and time of the image 63, and the user ID are also transmitted.

When the server 3 receives the image 63 and the text 65 and the like from the information terminal 1 (S17; receiving means), the server 3 records the image 63 and the text 65 and the like in the image DB 302 (S18; recording means).

When the text 65 is not embedded in the image 63 in S15 (when the image 63 and the text 65 are recorded separately), the image 63 is recorded in the image data area 51 of FIG. 4, and the text 65, the voice, the shooting date and time, etc. are recorded. It is recorded as additional information in the additional information area 52. Further, an image ID corresponding to the image 63 is generated, and this is also recorded in the additional information area 52 as additional information. When the text 65 is embedded in the image 63 in S15, the image 63 in which the text 65 is embedded is recorded in the image data area 51 as additional information, and other additional information is recorded in the additional information area 52.

The image 63 recorded in the image DB 302 can be viewed not only from the information terminal 1 of the user who is the sender of the image 63 but also by other users using the information terminal 1 of each user. However, if the user settings make it private, or if the public range is limited such that it is open only to specific friends, the image of that user is private or within the limited range. It will be open to the public at.

(2-2. Viewing and commenting images)
FIG. 10 is a flowchart for explaining the flow of processing in the image sharing system 10, and is a flowchart showing the flow for viewing and commenting on an image. In the explanation related to the following flow, a user other than the user who posted the image 63 shall view and comment on the image 63, and unless otherwise specified, the "user" is the other user. The "information terminal 1" is an information terminal 1 different from the information terminal 1 of the user who is the sender of the image 63. However, it is also possible for the user who posted the image 63 to view or comment on his / her own image 63 by the procedure described below.

The user starts an application program on the information terminal 1 (S21), inputs a user ID and the like, connects to the server 3, and sends a viewing request for the image 63 to the server 3 (S22).

Upon receiving the viewing request for the image 63 (S23), the server 3 reads the image 63 and the text 65 added to the image 63 from the image DB 302, and transmits the image 63 and the text 65 to the information terminal 1 (S24; transmission means). ).

In S24, when the text 65 is embedded in the image 63, the image 63 is transmitted, and when the image 63 and the text 65 are recorded separately (in each of the image data area 51 and the additional information area 52), the image 63 is transmitted. , The image 63 and the text 65 are transmitted as separate data. The server 3 temporarily records, for example, the user ID of the connected user and the image ID of the transmitted image 63.

When the information terminal 1 receives the image 63 and the text 65 from the server 3 (S25; receiving means), the information terminal 1 displays the image 63 and the text 65 on the display unit 13 (S26). 11A and 11B are examples of display screens at this time. FIG. 11A shows a case where the text 65 is embedded in the image 63, and FIG. 11B shows a case where the image 63 and the text 65 are recorded separately. Is an example of. In the latter, the text 65 is displayed in the vicinity of the image 63.

It is also possible to provide a speaker button (not shown) on the above screen, and when the speaker button is selected, the voice data corresponding to the text 65 can be acquired from the server 3 and reproduced. This makes it possible to share the live presence of the image 63 at the time of shooting among the users.

When posting a comment, the user selects the comment button 47 (see FIGS. 11A and 11B) on the screen and inputs the comment by operating the information terminal 1. The information terminal 1 transmits the input comment to the server 3 (S27).

When the server 3 receives the comment (S28), the server 3 records the comment in the comment DB 303 (S29).

As shown in FIG. 12, in the comment DB 303, the comment 80 is recorded in association with the image ID and the user ID (user A, user B, etc.) of the image 63. In the example of FIG. 12, the comment 80 is posted by the user A as "Swan, I saw it for the first time" to the image 63, and the comment 80 is posted by the user B as "I am also in the △△ pond." There is.

Comments can also be recorded by voice. For example, when the user selects the microphone button 42 on the screen shown in FIG. 11, the information terminal 1 can input voice and convert it into text by the same processing as in S13 and S14, thereby inputting a comment. can. The following processing is the same as after S27.

Further, as in the example of FIG. 9, it is also possible to post the same comment to a plurality of images 63 posted by the user (the user who is the sender of the image 63) at once. For example, by making it possible to display a list of images 63 posted in the past as thumbnail images in the same manner as in FIG. 9, and selecting a plurality of images 63 to post comments and performing the processes of S27 to S29, the plurality of images 63 can be displayed in a list. You can post the same comment 80 all at once.

When the comment is posted as described above, the next time the image 63 is viewed, the comment 80 associated with the image 63 in S24 is also transmitted to the user's information terminal 1 as additional information of the image 63, and FIG. The comment 80 is also displayed as shown in (a) and (b). FIG. 13A is an example in which the text 65 is embedded in the image 63, and the comment 80 is displayed below the image 63. FIG. 13B is an example in which the image 63 and the text 65 are recorded separately, the text 65 is displayed below the image 63, and the comment 80 is displayed below the text 65.

As described above, according to the present embodiment, the user can add the text 65 obtained by converting the input sound by inputting the sound to the image 63 taken by the information terminal 1 to the image. , The server can add this text 65 to the image 63 and provide it to other users. In the present embodiment, the input voice is converted into the text 65 and added to the image 63, so that the manual input of the text becomes unnecessary, and the user directly expresses a fresh impression, impression, and presence at the moment of shooting. The text 65 can be added to the image 63 so that other users can view it via the network.

Further, the text 65 can be displayed together with the image 63 on the information terminal 1 as an explanatory text or an impression of the image 63.

Further, since the information terminal 1 can add the same text 65 to a plurality of images 63 by one voice input, the same explanation such as "I went to Singapore" for a plurality of images 63 taken during a trip, for example. Etc. can be added.

However, the present invention is not limited to this. For example, in the present embodiment, the information terminal 1 converts the voice into text, but the conversion process from voice to text may be performed on the server 3 side.

In this case, the process of S14 is omitted, the audio data is recorded in the information terminal 1 in association with the image 63 in S15, and the audio data is transmitted to the server 3 as additional information of the image 63 in S16. The server 3 converts voice data into text and records it in the additional information area 52. By converting to text 65 on the server 3 side, the processing load of the information terminal 1 is reduced. On the other hand, when voice text conversion is performed on the information terminal 1 side as in the first embodiment, the user can correct an error or the like in the converted text 65.

Further, the information terminal 1 can arrange a plurality of images 63 and text 65 in a predetermined layout to create layout data, and can record this in S15.

For example, when a layout button (not shown) is provided in the operation area of the screen described with reference to FIG. 9, and the user selects a plurality of images 63 and touches the layout button, a layout screen (not shown) is displayed on the screen. A layout is selected according to the operation of the user, and layout data in which each image 63 and the text 65 added to each image 63 are arranged in the layout can be created. FIG. 14 is an example of this layout data, and by arranging a plurality of images 63 and text 65 along the layout, the plurality of images 63 and text 65 can be displayed on one screen like a photo book. can.

Subsequent processing is substantially the same as that described in FIGS. 5S16 to S18 and FIG. 10 (particularly, the description is based on the case where the text 65 is embedded in the image 63), and the layout data is stored in the image DB 302 of the server 3. It can be recorded and the layout data can be viewed by other users.

Although the preferred embodiments of the image sharing system and the like according to the present invention have been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or modified examples within the scope of the technical idea disclosed in the present application, and these also naturally belong to the technical scope of the present invention. Understood.

1: Information terminal 3, 9: SNS server 8: Network 10: Image sharing system 16: Camera 17: Mike 40: Image display area 63, 70: Image 65: Text 80: Comment

Claims (12)

An image sharing system that has multiple information terminals and a server.
The information terminal is
The camera that takes the image and
With a microphone that accepts voice input,
A display means for displaying a display screen including an image display area for displaying an image captured by the camera, a shutter button of the camera, and a microphone button operated when performing voice input by the microphone in the same screen. When,
A recording means for adding and recording the voice or a text obtained by converting the voice to the captured image as additional information.
A transmission means for transmitting the image and the additional information to the server, and
Have,
The server
A recording means for recording the image and recording the text obtained by converting the voice as additional information of the image.
A transmission means for transmitting the image and the text to the information terminal different from the information terminal from which the image is transmitted.
An image sharing system characterized by being equipped with. A save button is provided on the display screen.
The recording means adds the voice input by the operation of the microphone button or the text obtained by converting the voice to the image taken by the operation of the shutter button by the operation of the save button as additional information. The image sharing system according to claim 1. The image supply system according to claim 1 or 2, further comprising a time limit display means for displaying a time limit for voice input when inputting voice by operating the microphone button. The information terminal has a conversion means for converting the voice into text.
The image sharing system according to any one of claims 1 to 3 , wherein the transmission means of the information terminal transmits the image and the text to the server. The image sharing system according to claim 4 , wherein the information terminal displays both the image and the text. The image sharing system according to claim 5, further comprising a comment input means for receiving a voice input of a comment by providing a comment button and a microphone button in a screen displaying both the image and the text. The image sharing system according to any one of claims 4 to 6, wherein the information terminal creates layout data in which a plurality of the images and texts attached to the images are arranged in a predetermined layout. .. The image sharing system according to any one of claims 4 to 7, wherein the information terminal can set the same text for a plurality of images by one voice input. The server in the image sharing system according to any one of claims 1 to 8 . An information terminal connected to a server via a network.
The camera that takes the image and
With a microphone that accepts voice input,
A display means for displaying a display screen including an image display area for displaying an image captured by the camera, a shutter button of the camera, and a microphone button operated when performing voice input by the microphone in the same screen. When,
A recording means for adding and recording the voice or a text obtained by converting the voice to the captured image as additional information.
A transmission means for transmitting the image and the additional information to the server, and
A receiving means for receiving an image transmitted from a different information terminal to the server and a text added as additional information to the image from the server.
An information terminal characterized by being equipped with. A server connected to a plurality of information terminals according to claim 10 via a network,
A receiving means for receiving an image and a voice added as additional information to the image or a text obtained by converting the voice from the information terminal.
A recording means for recording the image and recording the text obtained by converting the voice as additional information of the image.
A transmission means for transmitting the image and the text to the information terminal different from the information terminal from which the image is transmitted.
A program to function as. An information terminal that has a camera that captures images and a microphone that accepts audio input, and is connected to a server via a network.
A display means for displaying a display screen including an image display area for displaying an image captured by the camera, a shutter button of the camera, and a microphone button operated when performing voice input by the microphone in the same screen. When,
A recording means for adding and recording the voice or a text obtained by converting the voice to the captured image as additional information.
A transmission means for transmitting the image and the additional information to the server, and
A receiving means for receiving an image transmitted from a different information terminal to the server and a text added as additional information to the image from the server.
A program to function as.

Images