JP2022114907A - Display device, control method and image formation device - Google Patents

Abstract

To provide a display device, etc., capable of outputting speech relating to a display screen when a utilized language and a language applied for the display screen are different from each other.SOLUTION: A display device includes a display unit, a speech output unit, a control unit and a storing unit. The storing unit stores a first language set by a user. The control unit displays a display screen on the display unit, obtains a screen creation rule applied to the display screen, and outputs, from the speech output unit, the speech of the first language corresponding to the screen creation rule when a second language applied for the display screen and the first language are different from each other.SELECTED DRAWING: Figure 6

Description

The present disclosure relates to display devices and the like.

2. Description of the Related Art Conventionally, so-called multifunction peripherals having document printing functions, facsimile functions, scanning functions, etc. have been widely used. Some MFPs have a function of switching the language to be used, and such MFPs display a screen using the language switched by the user. Thereby, the user can perform a predetermined operation through the screen displayed in an easy-to-understand language.

In addition, as a technology related to language switching, when displaying external content data provided by an external server on the operation screen, language switching on the server side and language switching on the multifunction device side are linked. Techniques have been proposed (see Patent Document 1, for example). As a result, the external content data can be displayed in the language used by the user in the image forming apparatus, and the external content data can be displayed in the language type of the external content data when the cooperation is not possible.

JP 2007-058632

However, there is a problem that when content data displayed in a language that is difficult for the user to understand is used, the user cannot perform appropriate operations. Patent Document 1 does not consider such a problem.

In view of the above-described problems, the present disclosure aims to provide a display device or the like capable of outputting sound related to the screen according to the language used.

In order to solve the above-described problems, the display device of the present disclosure includes a display unit, an audio output unit, a control unit, and a storage unit, wherein the storage unit is set by a user. A first language is stored, and the control unit displays a display screen on the display unit, acquires a screen creation rule applied to the display screen, and stores a second language used for the display screen. and the first language is different from the first language, the voice output unit outputs voice in the first language corresponding to the screen creation rule.

The control method of the present disclosure comprises the steps of: storing a first language set by a user; displaying a display screen; acquiring screen creation rules applied to the display screen; outputting a voice in the first language corresponding to the screen creation rule when the second language used in the screen is different from the first language.

An image forming apparatus according to the present disclosure is an image forming apparatus including a control unit, an audio output unit, a display unit, and an image forming unit, wherein the control unit sets a first language and provides content. display on the display unit a provision screen for providing the information, obtain a screen creation rule applied to the provision screen, and obtain a second language used in the provision screen and the first language in the first language corresponding to the screen creation rule is output from the audio output unit, and when the content is selected from the provided screen, the selected content is output from the image forming unit. It is characterized by outputting.

According to the present disclosure, it is possible to output screen-related audio according to the language used.

It is a figure which shows the outline|summary of the system in 1st Embodiment. 2 is a diagram for explaining the functional configuration of the image forming apparatus according to the first embodiment; FIG. FIG. 4 is a diagram for explaining an example of guidelines; 4 is a diagram for explaining the data structure of a voice correspondence table in the first embodiment; FIG. 4 is a diagram for explaining the flow of main processing in the first embodiment; FIG. It is a figure for demonstrating the flow of the guidance process in 1st Embodiment. It is a figure for explaining an example of operation in a 1st embodiment. It is a figure for explaining an example of operation in a 1st embodiment. It is a figure which shows the example of the audio|voice correspondence table in 2nd Embodiment. It is a figure for explaining the flow of guidance processing in a 2nd embodiment. FIG. 12 is a diagram for explaining the flow of main processing in the third embodiment; It is a figure for demonstrating the flow of the guidance process in 4th Embodiment. It is a figure for demonstrating the flow of the guidance process in 5th Embodiment.

An embodiment for carrying out the present disclosure will be described below with reference to the drawings. The following embodiments are examples for explaining the present disclosure, and the technical scope of the invention described in the claims is not limited to the following descriptions.

[1. First Embodiment]
[1.1 System Overview]
A system 1 according to the present embodiment will be described with reference to FIG. As shown in FIG. 1, a system 1 of this embodiment includes an image forming apparatus 10 and a server apparatus 20 that provides content. The image forming apparatus 10 and the server device 20 are connected via a network NW. Although the network NW is described as an external network such as the Internet, it may be a network other than the Internet (for example, a LAN (Local Area Network)) as long as each device can exchange information with each other.

The image forming apparatus 10 is an apparatus to which the display device of the present disclosure is applied. The image forming apparatus 10 acquires content stored in the server apparatus 20 by, for example, operating the image forming apparatus 10 by a user. As an example of the image forming apparatus 10, an MFP (Multi-Function Printer/Peripheral) having multiple functions such as a printer, a copier, and a FAX will be described.

The image forming apparatus 10 may receive the content from the server device 20 as necessary, or may store the content in advance. Note that in the present embodiment, a case where the image forming apparatus 10 acquires content in advance will be described.

[1.2 Configuration]
The configuration of the image forming apparatus 10 of this embodiment will be described with reference to FIG. As shown in FIG. 2, the image forming apparatus 10 includes a control unit 100, an image input unit 110, an image forming unit 120, a display unit 130, an operation unit 140, an audio output unit 150, and a storage unit 160. , and a communication unit 180 .

The control unit 100 controls the entire image forming apparatus 10 . The control unit 100 implements various functions by reading and executing various programs stored in the storage unit 160, and is configured by one or a plurality of arithmetic units (for example, CPU (Central Processing Unit)). there is

The image input unit 110 reads a document and inputs data of the read image. For example, the image input unit 110 is configured by a scanning device or the like equipped with a device such as a CIS (Contact Image Sensor) or a CCD (Charge Coupled Device) that converts optical information into electrical signals. Read the placed document. Note that the image input unit 110 may be configured by an interface (terminal) for reading an image stored in a USB (Universal Serial Bus) memory.

The image forming unit 120 forms (prints) an image on a recording medium such as recording paper. The image forming unit 120 is configured by, for example, a laser printer using an electrophotographic method.

The display unit 130 displays various information. The display unit 130 is configured by a display device such as an LCD (Liquid crystal display), an organic EL (electro-luminescence) panel, a micro LED (Light Emitting Diode) display, or the like.

Operation unit 140 receives an operation by a user who uses image forming apparatus 10 . The operation unit 140 is configured by input devices such as hardware keys (physical keys, numeric keys) and touch sensors. A general detection method such as a resistive film method, an infrared method, an electromagnetic induction method, or an electrostatic capacitance method may be used as a method for detecting an input in the touch sensor. Note that the image forming apparatus 10 may be equipped with a touch panel in which the display unit 130 and the operation unit 140 are integrally formed.

The audio output unit 150 outputs audio. The audio output unit 150 is composed of, for example, a speaker and an external output terminal to which an external speaker can be connected.

Note that the display unit 130, the operation unit 140, and the audio output unit 150 may be physical devices. For example, a display device, an operation device, and an audio output device may be connected using an input/output interface such as USB, DVI, or HDMI (registered trademark).

The storage unit 160 stores various programs and data necessary for the operation of the image forming apparatus 10 . The storage unit 160 is configured by, for example, a storage device such as an SSD (Solid State Drive), which is a semiconductor memory, or an HDD (Hard Disk Drive).

In this embodiment, the storage unit 160 stores set language information 162 and an audio correspondence table 166, and further includes a screen information storage area 164, an audio file storage area 168, and a content storage area 170 as storage areas. Secure.

The set language information 162 is information indicating a language (set language) used in the image forming apparatus 10 . The set language information 162 is, for example, information such as “Japanese” and “English”.

The screen information storage area 164 stores screen information, which is information used for screen display. In this embodiment, the control unit 100 reads predetermined screen information from the screen information storage area 164 and displays a screen based on the read screen information on the display unit 130 to display the predetermined screen.

The screen information may be, for example, any of the following information.
(1) A file representing a screen with predetermined elements. For example, an HTML (HyperText Markup Language) file, a CSS (Cascading Style Sheets) file, and an image file displayed on the screen.
(2) An image file showing the entire screen and a file showing an area for receiving a touch operation from the user.

The screen information is stored for each screen, and is associated with a predetermined screen by the file name of the screen information, the screen ID included in the screen information, and the like. Further, the screen information is stored for each language used for the screen displayed based on the screen information (hereinafter referred to as display language). Information indicating the display language may be described in the file, such as the lang attribute of the HTML tag, or may be included in the file name. Further, the image forming apparatus 10 may store a table that associates screen information with the display language of the screen information.

In the present embodiment, the screen displayed on the display unit 130 is a screen for particularly prompting the user to input an operation. screens to be created.

A content provider creates a screen (providing screen) for providing content. The screen information of the provided screen created by the content provider may be stored in advance in the image forming apparatus 10 or may be stored in the server apparatus 20 . Further, the screen information of the provided screen may be stored in a storage area on another cloud, and the image forming apparatus 10 may refer to it as appropriate.

The provision screen includes a screen for searching for content, a screen for selecting content, and a screen for displaying details of content selected by the user and accepting purchase of content (content detail screen).

The content provider creates the offer screen according to the guidelines. Here, a guideline is a rule (common rule) for creating a screen. In particular, the guideline is a rule that does not depend on the details of the content, and relates to the appearance of the provided screen and user's operation. Examples of guidelines (common rules) are shown below.

(A) Screen creation rule This is a rule regarding the appearance of the provided screen. For example, layout, color scheme, shape and color of buttons to be used are defined.

Specific examples of screen creation rules are as follows.
(1) Use a specific color (for example, pink or red) for buttons that need to be pressed for operation.
(2) Make buttons that need to be pressed for operation have a specific shape (for example, a rectangle with rounded corners).
(3) Put a specific mark (for example, an asterisk) on a button that needs to be pressed for operation.
(4) If there are multiple options on one screen, put a number at the beginning.
(5) Button for returning to the previous screen ・A button for canceling an operation is placed at the upper right of the screen and marked with an "X".
(6) The button to proceed to the next screen is placed at the bottom right of the screen and marked with a "→".

(B) Operation rule This is a rule regarding operation of the provided screen. For example, touch gestures by the user and physical keys pressed by the user are defined.

Specific examples of operation rules are as follows.
(1) When a left/right swipe operation is performed on a content detail screen, the content that is before or next to the currently displayed content is displayed among the searched content.
(2) When a pinch-in/pinch-out operation is performed on the detail screen of the content, the image of the currently displayed content is enlarged or reduced and displayed.
(3) When a pinch-out (pinch-close) operation is performed with three or more fingers on the content detail screen, a content search result screen is displayed.
(4) When the "1" button on the numeric keypad is pressed on the content details screen, detailed information on the currently displayed content (for example, registration date, number of outputs, ranking, content of another variation or related content) number).
(5) When the "reset" button of the hardware key is pressed on the content detail screen, a content search result screen is displayed.

(C) Audio Rule This is a rule regarding audio to be received when the image forming apparatus 10 is equipped with an audio input device such as a microphone.

Specific examples of voice rules are as follows.
(1) When the voice "return" is input, the screen returns to the previous screen.
(2) When voices such as "print", "OK", and "purchase" are input on the content detail screen, it is assumed that an operation to purchase the displayed content has been performed.

For example, the administrator of the image forming apparatus 10 provides the content provider with the guideline and an implementation method for realizing the screen in accordance with the guideline, in order to have the content provider create the provided screen in accordance with the guideline. Provide compiled materials. An example of material provided to content providers is shown in FIG. The material shown in FIG. 3 includes reference numbers (eg, "G1"), elements displayed on the screen (eg, "purchase decision button"), and guidelines for the elements (eg, "Use a pink round-corner button", etc.) and implementation method (eg, "specify "b-1" for the class attribute", etc.).

The implementation method shown in Fig. 3 is an example that assumes that the provided screen is created by HTML, and specifies the tags to be used, the attributes and attribute values to be used, and the character strings to be included in the elements. . When placing predetermined elements (e.g., buttons, progress displays, options, etc.) on the provided screen, the content provider can provide the provided screen in accordance with the guidelines by writing HTML in accordance with the implementation method corresponding to the element. It can be displayed on the image forming apparatus 10 .

Note that the mounting method may specify the colors and shapes of elements used to display predetermined elements, the images to be used, and the types of marks to be displayed. Further, the implementation method may be specified on the premise that the provided screen is created from an image file. For example, the mounting method may specify the colors used to display predetermined elements, the shapes of the elements, the images to be used, the shapes of marks to be displayed, and the like. The content provider can cause the image forming apparatus 10 to display the provided screen in accordance with the guideline by creating an image file for displaying an image in accordance with the specification indicated in the implementation method.

In addition to the rules described above, the guidelines may include rules related to the entire screen, such as background, character size, and button size, as well as rules related to operations and rules related to input voice. may

On the other hand, the screen specific to the image forming apparatus 10 is a screen other than the screen created by the content provider. The screen specific to the image forming apparatus 10 is, for example, a menu screen displayed first when the user uses the image forming apparatus 10, or a copy function or a facsimile function, which are functions provided in the image forming apparatus 10 itself. This is a screen for setting the Further, the screen specific to the image forming apparatus 10 includes a purchase screen for purchasing content, a screen displayed during printing, a screen displayed when printing is completed, and the like.

The voice correspondence table 166 is a table in which voices corresponding to the attributes of the displayed screen (characteristics of the screen) are associated with each other. Note that in the present embodiment, the attribute of the screen associated with the sound (the attribute to be notified) is called the attribute for notification.

The voice correspondence table 166, for example, as shown in FIG. 4, corresponds to combinations of attributes for notification (for example, "specify "b-1" for class attribute") and languages (for example, "English"). Specify audio (eg, "purchase_en.mp3"). For the voices contained in the voice correspondence table 166, file names of voice files stored in the voice file storage area 168 are stored. Note that the audio correspondence table 166 may store the audio data itself instead of the file name of the audio file.

FIG. 4 is a diagram showing an example in which tags included in HTML, attributes of tags, and attribute values are designated as notification attributes on the premise that screen information is described in HTML. Note that the attributes for notification may be attributes other than the attributes described above. For example, the notification attribute may be a screen ID or characters displayed on the screen. Also, the notification attribute may be the color included in the screen, or the shape, position, and color of a predetermined element (for example, button) included in the screen.

In this way, the notification attribute defines the attribute of the screen created based on the guideline. Therefore, the notification attribute corresponds to the implementation method of the predetermined guideline.

A sound corresponding to a certain attribute for notification is a sound corresponding to a predetermined guideline, and includes sounds for each language such as English, Chinese, French, etc., as shown in FIG. In this embodiment, voices for each language corresponding to a predetermined guideline are referred to as a voice set.

The content storage area 170 stores content data provided by content providers.

The communication unit 180 communicates with other devices via a LAN (Local Area Network) or a WAN (Wide Area Network). The communication unit 180 is configured by, for example, a communication device such as a NIC (Network Interface Card) used in a wired/wireless LAN, and a communication module.

Control unit 100 functions as language setting unit 102 , guideline acquisition unit 104 , audio output control unit 106 , and image processing unit 108 by executing programs stored in storage unit 160 .

The language setting unit 102 sets a set language. The language setting unit 102 sets the set language to the default language when the user starts using the image forming apparatus 10, or sets the set language based on the user's operation of changing the set language. . Note that the language setting unit 102 stores information indicating the set language as the language setting information 162 . The default language is a language set in advance by an administrator or the like of the image forming apparatus 10. For example, the language used at the installation location (destination) may be set.

The guideline acquisition unit 104 acquires guidelines applied to the screen displayed on the display unit 130 .

For example, the guideline acquisition unit 104 acquires the following information from the screen displayed on the display unit 130.
(1) Tags used, their attributes, and their attribute values (2) Characters to be displayed and their display positions (3) Marks to be displayed and their display positions (4) Buttons to be displayed shape, color, position

Specifically, when the screen information is an HTML file or a CSS file, the guideline acquisition unit 104 acquires tags used, attributes of tags, characters to be displayed, marks, shapes, positions and colors of buttons, Get directly from HTML and CSS files.

Further, when the screen information is an image file, the guideline acquisition unit 104 analyzes the image based on the image file to obtain characters, marks, button shapes, characters, marks, and buttons included in the displayed screen. Get the display position of .

Note that the guideline acquisition unit 104 analyzes the event handlers and function names described in the HTML file, the source code of JavaScript (registered trademark) read via the HTML file, the library used, etc., to obtain the operation rule. and guidelines on voice rules.

The voice output control unit 106 outputs a description of the displayed screen by voice. Processing executed by the audio output control unit 106 will be described later.

The image processing unit 108 performs processing related to various images. For example, the image processing unit 108 executes sharpening processing and gradation conversion processing on the image read by the image input unit 110 .

[1.3 Flow of processing]
[1.3.1 Main processing]
Next, processing executed by the image forming apparatus 10 in this embodiment will be described. First, the main processing will be described with reference to FIG.

First, control unit 100 displays a menu screen on display unit 130 (step S102). For example, control unit 100 reads screen information of a menu screen corresponding to the set language from screen information storage area 164 and displays a screen based on the read screen information on display unit 130 .

Subsequently, when an operation to change the set language is performed, the control unit 100 (language setting unit 102) changes (sets) the set language based on the operation (step S104; Yes→step S106). An operation to change the set language is, for example, an operation in which the user selects a button for selecting a set language included in the menu screen and further selects one language.

For example, when the user selects a button for selecting a set language, the language setting unit 102 acquires languages that can be set as the set language, and displays a list on the display unit 130 . The languages that can be set as the setting language are stored in the storage unit 160 in advance as a list, for example, and are acquired by reading the list by the language setting unit 102 . Note that the language that can be set as the set language may be obtained by reading the display language from the screen information stored in the screen information storage area 164 by the language setting unit 102 . Then, the language setting unit 102 displays a list of languages that can be selected as the setting language on the display unit 130, and allows the user to select one language. Language setting unit 102 stores the language selected by the user in storage unit 160 as set language information 162 .

After changing (setting) the set language, the control unit 100 updates the menu screen (step S108). The process of step S108 is the same as the process of step S102.

Subsequently, the control unit 100 determines whether or not purchase of content (use of content service) is selected from the menu screen (step S110).

When content purchase is selected, a provision screen is displayed on the display unit 130 (step S110; Yes→step S112). As already described, the providing screen includes a content search screen, a content list display screen, a content detail screen, and the like. These screens are displayed according to the user's operation. The control unit 100 acquires screen information of the provided screen in order to display the provided screen. The screen information of the provided screen may be obtained by reading the screen information of the provided screen from the screen information storage area 164, or may be obtained by referring to the server device 20. FIG.

It should be noted that if content purchase is not selected, that is, if a function other than content purchase is selected from the menu screen, the control unit 100 executes a predetermined process according to the selected function ( Step S110; No).

Subsequently, the control unit 100 determines whether or not the purchase of the content has been determined (step S114). For example, the control unit 100 determines that the purchase of the content has been decided when the purchase decision button included in the details screen of the content is selected.

Subsequently, when content purchase is determined, the control unit 100 displays a content purchase screen on the display unit 130 (step S114; Yes→step S116). For example, the control unit 100 reads the screen information of the purchase screen corresponding to the set language from the screen information storage area 164 and displays the screen based on the read screen information on the display unit 130 .

Further, after the user performs an operation to purchase the content, the control unit 100 prints the content purchased by the user. At this time, the control unit 100 reads from the screen information storage area 164 the screen information of the screen to be displayed during printing and the screen to be displayed at the completion of printing according to the printing status, and displays the screen based on the read screen information. Displayed in section 130 .

Note that if the user does not decide to purchase the content, such as when the user performs an operation to reselect the content, the control unit 100 returns to step S112 and displays the provision screen again (step S114; No → step S112).

Subsequently, the control unit 100 determines whether or not the business has ended (step S118). The case where the business is closed is, for example, the case where the business end time set by the administrator of the image forming apparatus 10 has passed. If the business has not ended, the control unit 100 displays the menu screen by executing the process of step S102 again (step S118; No→step S102).

[1.3.2 Guidance processing]
Subsequently, a process (guidance process) for executing an explanation (guidance) of the screen displayed on the display unit 130 will be described with reference to FIG. Control unit 100 executes the guidance process at the timing when the screen displayed on display unit 130 is switched. This enables the control unit 100 to provide the user with guidance regarding the screen each time the screen changes.

First, the control unit 100 determines whether or not the set language is different from the display language (step S152). For example, the control unit 100 acquires the set language at the time of executing step S152 and the display language of the screen information of the screen displayed on the display unit 130 at the time of executing step S152, and determines whether they match. judge.

When the set language is different from the display language, the control unit 100 (guideline acquisition unit 104) acquires the guideline applied to the screen displayed on the display unit 130 (step S152; Yes→step S154). Through this processing, information such as the tags used in the displayed screen and the attributes and attribute values of the tags are acquired.

Subsequently, the control unit 100 (voice output control unit 106) determines whether or not the guideline acquired by the guideline acquisition unit 104 in step S154 has an attribute for notification (step S154). For example, if any of the guidelines (for example, tags, attributes and attribute values of tags) acquired in step S154 matches the attribute for notification stored in the audio correspondence table 156, the voice output control unit 106 notifies It is determined that there is an attribute for use.

If there is an attribute for notification, the control unit 100 (audio output control unit 106) determines an audio set corresponding to the attribute for notification (step S156; Yes→step S158). Note that when a plurality of guidelines match any of the notification attributes, the audio output control unit 106 determines a plurality of audio sets.

The guideline acquired in step S154 may include a guideline that does not match any of the notification attributes. In other words, the guidelines may include guidelines that are not subject to notification (for example, guidelines unrelated to user operations, guidelines regarding background colors). For such guidelines, the audio output control unit 106 does not determine the audio set.

Subsequently, the control unit 100 (speech output control unit 106) acquires the speech corresponding to the set language from the speech set determined in step S158 (step S160). For example, the audio output control unit 106 acquires the file name of the audio file of the audio corresponding to the set language from the audio set.

Subsequently, the control unit 100 (audio output control unit 106) outputs the audio acquired in step S160 via the audio output unit 150 (step S162). For example, in step S160, when the file name has been acquired, the audio output control unit 106 reads the audio file corresponding to the acquired file name from the audio file storage area 168, and outputs the audio based on the audio file. It controls the output from the output unit 150 .

Through the above-described processing, if the set language and the display language are different, the guidance regarding the displayed screen is output by voice in the language corresponding to the set language. Thereby, when the user changes the set language, the user can listen to the voice guidance in the language corresponding to the changed set language.

In step S152, if the set language matches the display language, control unit 100 omits the processes from step S154 to step S160. Thereby, when the set language and the display language match, it is possible to prevent the sound from being played (omit the execution of the guidance).

The guidance process described above is executed, for example, at the timing when the screen displayed on the display unit 130 is switched to the provision screen. As a result, even if the provided screen does not support the set language, the audio corresponding to the guideline applied to the provided screen and using the set language is output.

Note that the guidance process may also be executed at the timing when the screen specific to the image forming apparatus 10 is switched. In this case, the audio correspondence table 166 is stored with a notification attribute (for example, screen ID) related to a screen unique to the image forming apparatus 10, and the audio file storage area 168 is stored with the audio corresponding to the notification attribute. Since a screen corresponding to the set language is displayed as a screen specific to the image forming apparatus 10, the set language and the display language are not different. Therefore, as guidance processing when a screen unique to image forming apparatus 10 is displayed, control unit 100 omits determination in step S152 and executes processing from step S154 to step S162.

[1.4 Operation example]
An operation example in this embodiment will be described with reference to FIGS. 7 and 8. FIG. 7 and 8 are diagrams showing the transition of the screen displayed on the display unit 130 and the content of the voice (guidance) output when the display is switched to the screen.

It should be noted that in the description of the operation example, the initial setting of the setting language is assumed to be Japanese. Also, the image forming apparatus 10 will be described as executing the guidance process at the timing when the screen displayed on the display unit 130 is switched to a screen specific to the image forming apparatus 10 or a provided screen.

Also, in the description of the operation example, it is assumed that the provided screen is supported only in Japanese.

A display screen W100 in FIG. 7 shows a screen displayed and a voice (guidance) output when the user starts using the image forming apparatus 10 . Since the default setting language is Japanese, the menu screen in Japanese is displayed as the display screen W100, and the Japanese voice T100 is output. As the sound corresponding to the menu screen, for example, a sound indicating selection of a service is output. Here, if the user cannot understand the explanation in Japanese included in the display screen W100, the user selects the button B100 for changing the set language.

A display screen W<b>110 in FIG. 7 is a screen for selecting a setting language, and is a screen specific to the image forming apparatus 10 . The display screen W110 displays a list of languages that can be selected by the user as the set language. Also, as guidance, a voice T110 indicating selection of a language is output. When one language is selected on the display screen W110, the selected language is set as the set language.

A display screen W120 in FIG. 7 is an English menu screen displayed when the user selects English on the display screen W110. Also, an English voice T120 is output as guidance. In this manner, the screen specific to the image forming apparatus 10 can switch the language displayed on the screen based on the user's operation.

A display screen W130 in FIG. 8 is a content detail screen displayed when the user selects purchase of content and further selects specific content on the display screen W120.

The content detail screen is an offer screen created by the content provider and only Japanese is supported. Therefore, even if the set language is changed to English, the provision screen using English is not displayed. In other words, the language displayed on the screen cannot be switched based on the user's operation.

Here, when the provision screen is created according to the guidelines shown in FIG. 3, the purchase determination button B130 included in the content detail screen is displayed according to the guidelines. For example, the purchase decision button B130 is a pink button with rounded corners, and the button displays the words "purchase".

Here, the image forming apparatus 10 outputs English voice (guidance) corresponding to the guideline applied to the display screen W130. For example, as guidance, the image forming apparatus 10 outputs an English voice T130 explaining that content can be purchased by selecting a pink button.

By listening to the voice T130, the user can understand that the pink button is for purchasing the content. A pink purchase decision button B130 is also displayed on the display screen W130. Therefore, even if the user cannot understand the characters "Purchase" displayed on the purchase decision button B130, the user can understand that the purchase decision button B130 is a button for purchasing the content.

A display screen W140 in FIG. 8 is a purchase screen displayed when the user selects the purchase decision button B130 on the display screen W130. The purchase screen is a screen specific to the image forming apparatus 10 . Therefore, a screen using English, which is the set language, is displayed. Also, as guidance corresponding to the purchase screen, for example, an English voice T140 indicating that the content will be printed by selecting the yellow button is output.

A display screen W150 in FIG. 8 is displayed while the content is being printed. A display screen W160 in FIG. 8 is a screen displayed when printing of the content is completed. These screens are screens unique to the image forming apparatus 10, and screens using English, which is the set language, are displayed.

After changing the set language to English in this way, the guidance on the displayed screen is output in English at the timing when the screen is switched. Thereby, even if the display language is different from the set language, the user can listen to the guidance in the set language. Also, the screen is created according to a predetermined guideline, and guidance corresponding to the guideline used on the screen is output, so that the user can operate the screen by listening to the guidance.

When a text message is included in the provided screen provided by the content provider, the control unit 100 may translate and display the text message. For example, when an attribute for translation is added as an attribute, the image forming apparatus 10 translates the text into the set language text using an external server or a translation function. Also, the control unit 100 may output a voice corresponding to the text message translated into the set language.

According to this embodiment, the buttons and the like are displayed in a distinguishable manner by the colors of the marks and buttons laid out on the screen according to the predetermined guidelines, and the voice corresponding to the specific guidelines is displayed in a language that the user can understand. output. As a result, the image forming apparatus 10 can assist the user's operation with voice, improve the user's convenience, and prevent erroneous operations due to language differences.

Also, by unifying the guidelines, it is possible to minimize the pattern of voice guidance even when a plurality of content providers create screens.

[2. Second Embodiment]
Next, a second embodiment will be described. The second embodiment is an embodiment that provides simplified guidance when the set language and the display language match in guidance processing.

Simplified guidance refers to guidance in which information to be provided is suppressed compared to guidance that is output when the set language and display language do not match. In this embodiment, the image forming apparatus 10 implements one of the three methods described below.

(1) Method of not including voices of a predetermined language in some voice sets This method is a method of not including voices of a predetermined language in some voice sets in the voice correspondence table 166 . For example, as shown in FIG. 9(a), in the audio set corresponding to the notification attribute "class attribute with "b-2"specified" and "class attribute with "b-3"specified" , do not include Japanese audio.

When the image forming apparatus 10 implements this method, as guidance processing, the control unit 100 omits the processing shown in step S152 among the processing shown in FIG. 6, and executes the processing from step S154 to step S162. . Accordingly, even when the set language and the display language match, the control unit 100 (audio output control unit 106) determines the audio set when there is an attribute for notification.

At this time, even if there is an attribute for notification such as "class attribute with "b-2" specified" or "class attribute with "b-3" specified", if the display language is Japanese , the sound corresponding to these attributes for notification is not acquired. Therefore, when the display language is Japanese, the types of sounds to be output are suppressed compared to when the display language is a language other than Japanese. Thereby, the control unit 100 can output simplified guidance.

(2) A method of varying the content of the audio included in the audio set according to language It is a method to let Thereby, the content of the output guidance can be made different between the language that matches the display language and the language that does not match the display language. Specifically, when the display language cannot be set to anything other than Japanese, as shown in FIG. Make it a voice for outputting the omitted content.

When the image forming apparatus 10 implements this method, the control unit 100 omits the process shown in step S152 among the processes shown in FIG. The processing up to S162 is executed. Accordingly, when the display language is Japanese, the control unit 100 can output simplified guidance compared to when the display language is another language.

(3) Method of varying the voice output range This method includes voices that include all the contents of the guidance in the voice set, and then outputs voices according to whether the set language and the display language match. This is a method of varying the range. Thereby, the content of the output guidance can be made different between the language that matches the display language and the language that does not match the display language. Specifically, when the display language cannot be set to anything other than Japanese, the control unit 100 outputs only a part of the range of the Japanese voice when outputting the Japanese voice, and outputs the voice other than Japanese. If so, output the entire range of the sound. By doing so, once the voice including the content of the guidance is prepared, the output range of the voice is automatically determined by the control unit 100 according to whether the display language and the set language match. be.

When the image forming apparatus 10 implements this method, the audio set includes audio corresponding to a set language that can be set by the user, as shown in FIG. 9(c).

On the other hand, the control unit 100 executes the process shown in FIG. 10 as the guidance process. 10, the same reference numerals are given to the same processes as those shown in FIG. In the guidance process in this method, control unit 100 executes the process from step S154 to step S160 regardless of whether the set language is different from the display language.

Subsequently, the control unit 100 determines whether or not the set language and the display language are different. If the set language and the display language are different, the voice acquired in step S160 is output as it is (step S202; Yes→step S162). In this case, the control unit 100 can output all the contents of the guidance.

On the other hand, when the set language and the display language are the same, the control unit 100 outputs a part of the voice acquired in step S160 (step S202; No → step S204). For example, the control unit 100 outputs audio by reproducing only a part of the reproduction section of the audio file acquired in step S160. A part of the playback section may be automatically determined by the control unit 100 based on the waveform data indicated by the audio file, for example. Note that information indicating a partial playback interval for each audio file may be stored in the storage unit 160 in advance, and the control unit 100 may determine by referring to the information indicating the partial playback interval. In this case, the control unit 100 can output a part of the contents of the guidance.

According to this embodiment, when the set language and the display language match, the image forming apparatus can partially simplify the guidance and output it. As a result, the image forming apparatus can prevent the user from feeling annoyed by the execution of the guidance when the screen using the language that the user can understand is displayed.

[3. Third Embodiment]
Next, a third embodiment will be described. The third embodiment is an embodiment in which a screen provided by a content provider supports multiple languages. This embodiment replaces FIG. 5 of the first embodiment with FIG. Note that the same functional units and processes are denoted by the same reference numerals, and descriptions thereof are omitted.

The flow of the main processing of this embodiment will be described with reference to FIG. In this embodiment, when purchase of content (use of content service) is selected, the control unit 100 determines whether or not a provision screen using a language corresponding to the set language can be displayed (step S302). ). For example, when the screen information of the provided screen using the language corresponding to the set language can be acquired from the screen information storage area 164 or the server device 20, the control unit 100 displays the provided screen using the language corresponding to the set language. Determine that it is possible.

When the provided screen using the language corresponding to the set language can be displayed, the control unit 100 acquires the screen information of the provided screen using the language corresponding to the set language, and displays the provided screen on the display unit 130. (step S302; Yes→step S304). If the purchase of the content is not determined, the control unit 100 returns to step S304 and again displays the provision screen using the language corresponding to the set language on the display unit 130 (step S114; No → step S304).

On the other hand, when the provision screen using the language corresponding to the set language cannot be displayed, the control unit 100 selects the language used for the provision screen (step S302; No→step S306). For example, the control unit 100 allows the user to select one language by displaying a list of languages supported on the provision screen, and selects the language selected by the user as the language used for the provision screen.

Subsequently, the control unit 100 acquires the screen information of the provided screen using the language selected by the user, and displays the provided screen on the display unit 130 (step S308). If the purchase of the content is not determined, the control unit 100 returns to step S308 and again displays the provision screen using the language selected by the user on the display unit 130 (step S310; No → step S308).

As for the guidance process, in the present embodiment, the control unit 100 executes the guidance process at the timing when the screen displayed on the display unit 130 is switched, as in the first embodiment.

Further, in the guidance processing of the present embodiment, when the set language is changed and the provision screen corresponding to the set language cannot be displayed, execution of the guidance may be omitted (suppressed). Accordingly, when the setting language selected by the user is not supported on the provision screen, the image forming apparatus 10 can suppress the output of the sound based on the mismatch between the setting language and the display language.

According to the above-described processing, the image forming apparatus 10 performs the following operations in the present embodiment.

(1) When the provided screen supports the set language If the set language is changed to a predetermined language (for example, English) in step S106 and the provided screen supports the predetermined language, the predetermined language is used. The provided screen is displayed.

Moreover, since the set language and the display language match at the timing when the provision screen is displayed, the execution of the guidance may be omitted or partially simplified guidance may be executed in the guidance processing.

(2) When the provided screen does not support the set language If the set language is changed to a predetermined language (for example, Indonesian) in step S106 and the provided screen does not support the predetermined language, the user purchases the content. After performing the operation of , select the language used on the provided screen.

In this case, the user can select a predetermined language (for example, English) as the language used in the provided screen.

Here, the set language is different from the display language on the provided screen. Therefore, the image forming apparatus 10 of the present embodiment can omit execution of guidance or execute guidance in the set language.

According to the present embodiment, when a screen other than a screen specific to the image forming apparatus supports a plurality of languages, the image forming apparatus appropriately executes guidance according to the set language and the display language. can be suppressed.

[4. Fourth Embodiment]
Next, a fourth embodiment will be described. The fourth embodiment is an embodiment capable of appropriately executing guidance or suppressing execution of guidance when the image forming apparatus does not support voice corresponding to the set language. This embodiment replaces FIG. 6 of the first embodiment with FIG. Note that the same functional units and processes are denoted by the same reference numerals, and descriptions thereof are omitted.

The flow of guidance processing according to the present embodiment will be described with reference to FIG. In this embodiment, after determining the audio set, the control unit 100 (audio output control unit 106) determines whether or not there is audio in the set language (step S452).

If there is a voice of the set language in the voice set, the control unit 100 (voice output control unit 106) acquires the voice (step S452; Yes→step S160).

On the other hand, if there is no voice of the set language in the voice set, the voice corresponding to the general language is obtained from the voice set (step S452; No→step S454). A general-purpose language is a language set by an administrator or the like of the image forming apparatus 10, and is a language to be used for voice output instead when the voice corresponding to the set language is not supported. An administrator or the like of the image forming apparatus 10 sets a language generally used at the installation location (destination) of the image forming apparatus 10 or English, which is widely used worldwide, as a general language.

Also, the control unit 100 (audio output control unit 106) outputs the audio acquired in step S454 via the audio output unit 150 (step S162). Thereby, the control unit 100 (voice output control unit 106) can execute voice guidance using a general-purpose language.

If there is no voice in the set language in step S452, the control unit 100 (voice output control unit 106) omits the processing of steps S454 and S162 in FIG. Thus, the guidance may be omitted (suppressed). By omitting the guidance, when both the display language and the language used for the guidance are difficult for the user to understand, the image forming apparatus 10 may confuse the user by outputting the guidance. can be prevented.

If there is no audio in the set language in step S452, the control unit 100 (audio output control unit 106) displays a list of languages that have audio in the set language in the audio set, prompts the user to select one language, and prompts the user to select one language. Voice guidance may be provided in a language selected by the

According to the present embodiment, it is possible to appropriately execute guidance or suppress the execution of guidance when the image forming apparatus does not support voice corresponding to the set language.

[5. Fifth Embodiment]
Next, a fifth embodiment will be described. The fifth embodiment is an embodiment capable of executing appropriate guidance when a screen that has already been displayed is displayed again. This embodiment replaces FIG. 6 of the first embodiment with FIG. Note that the same functional units and processes are denoted by the same reference numerals, and descriptions thereof are omitted.

The flow of guidance processing according to the present embodiment will be described with reference to FIG. In the present embodiment, if there is an attribute for notification (step S156; Yes), the control unit 100 (audio output control unit 106) determines the audio set and then displays the displayed screen on the display unit 130. It is determined whether or not there is (step S158→step S552).

Here, in order to determine whether or not a screen that has already been displayed is displayed on the display unit 130, for example, when a certain user starts using the image forming apparatus 10, the audio output control unit 106 The screen ID of the screen displayed on the display unit 130 is stored based on the operation of . In this case, if the screen ID of the screen being displayed on the display unit 130 matches any of the screen IDs of the displayed screens in step S452, the audio output control unit 106 displays the displayed screen. determined to be

When the displayed screen is displayed, control unit 100 (audio output control unit 106) displays a re-guidance button for inputting an instruction to execute re-guidance on the screen displayed on display unit 130. (step S552; Yes→step S554). It should be noted that re-guidance in the present embodiment means guidance that is not simplified.

Further, the control unit 100 (speech output control unit 106) acquires the set language speech from a part of the speech sets determined in step S158 (step S556).

For example, numerical values and categories such as importance and priority are associated in advance for each notification attribute, and the audio output control unit 106 determines the audio set from which audio is to be acquired according to the importance and priority. do. More specifically, when a screen that has already been displayed is displayed on the display unit 130, the audio output control unit 106 increases the threshold of importance and priority of the audio to be acquired according to the number of times it has been displayed. , the greater the number of times it is displayed, the more difficult it is for an audio set with low importance or priority to be selected. By doing so, the voice output control unit 106 can gradually decrease the number of voices to be acquired when the same screen is displayed.

Subsequently, the control unit 100 (audio output control unit 106) outputs the audio acquired in step S556 via the audio output unit 150 (step S558).

Subsequently, control unit 100 (voice output control unit 106) determines whether or not the re-guidance button displayed in step S554 has been selected (step S560). If the re-guidance button is selected (step S560; Yes), the control unit 100 (voice output control unit 106) executes steps S160 and S162.

Here, in step S160, the voice output control unit 106 acquires voices corresponding to the set language from all voice sets determined in step S158. Therefore, the voice output control unit 106 can execute unsimplified guidance in step S162.

According to the image forming apparatus of the present embodiment, guidance can be shortened when the same screen is repeatedly displayed. Further, when the user performs an operation to confirm the guidance, the image forming apparatus according to the present embodiment outputs complete guidance corresponding to the displayed screen, thereby allowing the user to confirm the screen. As described above, according to the present embodiment, the image forming apparatus can provide appropriate guidance on screens that are displayed multiple times.

[6. Sixth Embodiment]
The sixth embodiment is an embodiment in which the display device described in the first to fifth embodiments is provided in an apparatus other than the image forming apparatus.

The apparatus according to this embodiment includes a display unit 130, an operation unit 140, and an audio output unit 150. The storage unit 160 stores set language information 162 and an audio correspondence table 166, and further includes a screen information storage area 164 and an audio file. A storage area 168 is secured.

The apparatus according to this embodiment also includes a control unit 100 , which functions as a language setting unit 102 , a guideline acquisition unit 104 and an audio output control unit 106 .

By executing the processes described in the first to fifth embodiments in the apparatus described above, it is possible to output guidance when the set language and the display language do not match.

The present embodiment can be applied to devices such as electronic blackboards used in meeting rooms, signage devices installed in commercial facilities, and vending machines, for example. It can also be applied to information processing devices such as smartphones and PCs (Personal Computers). In this case, the guidance of the screen displayed by the program executed in the information processing device is output by voice.

As described above, according to the present embodiment, it is possible to execute appropriate guidance even in apparatuses other than the image forming apparatus when the set language and the display language are different.

[7. Modification]
The present invention is not limited to the embodiments described above, and various modifications are possible. That is, the technical scope of the present invention also includes embodiments obtained by combining technical means appropriately modified within the scope of the present invention.

In addition, although the above-described embodiments are described separately for convenience of explanation, it is of course possible to combine them within the technically possible range. For example, the third embodiment and the fourth embodiment may be combined. In this case, when the screen provided by the content provider supports multiple languages and does not support voice corresponding to the set language, the image forming apparatus can execute guidance in the general language. .

In addition, the program that operates in each device in the embodiment is a program that controls the CPU and the like (a program that causes the computer to function) so as to implement the functions of the above-described embodiments. The information handled by these devices is temporarily stored in a temporary storage device (for example, RAM) during processing, and then stored in storage devices such as various ROMs (Read Only Memory) and HDDs. The data is read, corrected, and written by the CPU.

Examples of recording media for storing programs include semiconductor media (eg, ROM, non-volatile memory cards, etc.), optical recording media/magneto-optical recording media (eg, DVD (Digital Versatile Disc), MO (Magneto Optical Disc), MD (Mini Disc), CD (Compact Disc), BD (Blu-ray (registered trademark) Disc), etc.), magnetic recording media (e.g., magnetic tape, flexible disc, etc.), etc. . By executing the loaded program, the functions of the above-described embodiments are realized. In some cases, inventive features are realized.

Also, when distributed to the market, the program can be stored in a portable recording medium for distribution, or transferred to a server computer connected via a network such as the Internet. In this case, of course, the storage device of the server computer is also included in the present invention.

10 image forming apparatus 100 control unit 102 language setting unit 104 guideline acquisition unit 106 audio output control unit 108 image processing unit 110 image input unit 120 image forming unit 130 display unit 140 operation unit 150 audio output unit 160 storage unit 162 set language information 164 Screen information storage area 166 Audio correspondence table 168 Audio file storage area 170 Content storage area 180 Communication unit

Claims (8)

A display device comprising a display unit, an audio output unit, a control unit, and a storage unit,
The storage unit stores a first language set by a user,
The control unit
displaying a display screen on the display unit;
Acquiring a screen creation rule applied to the display screen,
When the second language used for the display screen is different from the first language, the audio output unit outputs audio in the first language corresponding to the screen creation rule. display device. When the first language and the second language are different, and if the screen creation rule includes an attribute for notification, the control unit is configured to output voice in the first language corresponding to the attribute for notification. 2. The display device according to claim 1, wherein the audio output unit outputs: . 3. The display device according to claim 1, wherein when the control unit cannot output the sound of the first language, the sound of the third language is output. When the first language and the second language match, the control unit omits part or all of the audio output of the first language corresponding to the screen creation rule. 4. The display device according to any one of claims 1 to 3, characterized in that: When the display screen using the first language cannot be displayed, the control unit causes the user to select the second language that can be displayed,
The display device according to any one of claims 1 to 4, wherein the display screen is displayed using the selected second language. 6. The control unit displays a re-guidance button on the display screen when the display screen displayed on the display unit is already displayed. The display device according to the item. storing a first language set by a user;
displaying a display screen;
obtaining screen creation rules applied to the display screen;
a step of outputting audio in the first language corresponding to the screen creation rule when the second language used in the display screen is different from the first language;
A control method comprising: An image forming apparatus comprising a control unit, an audio output unit, a display unit, and an image forming unit,
The control unit
set the first language,
displaying a provision screen for providing content on the display unit;
Acquiring the screen creation rule applied to the provided screen,
when the second language used in the provided screen is different from the first language, outputting audio in the first language corresponding to the screen creation rule from the audio output unit;
An image forming apparatus, wherein when the content is selected from the providing screen, the selected content is output from the image forming unit.

Images