JP6551363B2 - Information processing device - Google Patents

Description

  The present invention relates to an information processing apparatus.

  Techniques related to processing of image information, for example, processing of displaying image information are disclosed in Japanese Patent Application Laid-Open No. 2012-3442 (Patent Document 1) and Japanese Patent Application Laid-Open No. 2013-114467 (Patent Document 2).

  The exhibit information presentation device disclosed in Patent Literature 1 selectively presents information about a specific exhibit to which a viewer has paid attention among a plurality of exhibits. The information presentation apparatus for exhibit stores an arrangement position information storage unit storing arrangement position information indicating arrangement positions in a three-dimensional space for a plurality of exhibits, and stores presentation target information for a plurality of exhibits, respectively. A position designation tool having a function of pointing at a predetermined reference point in a three-dimensional space by a placement operation by a viewer, and a structure for pointing a point to a presentation target information storage unit Position detection means for detecting the position of the predetermined reference point determined and the position of the reference point pointed by the position specification tool, "position of reference point" detected by the position detection means, and "reference point" Gaze direction predicting means for predicting the gaze direction estimated that the viewer is turning attention based on the “position of the user” and the arrangement position information stored in the arrangement position information storage unit. The display object information about the gaze display object is read from the gaze display object specifying means for specifying the display object arranged in the gaze direction or the vicinity as the gaze display object, and the information is browsed Information presenting means for presenting to a person.

  In addition, the display system disclosed in Patent Document 2 includes an operation detection unit that detects a user operation on a target area in real space, and a target copy operation performed in response to a user's predetermined copy operation on the target area. An image acquisition unit for acquiring a range designated by a copy operation in an image obtained by imaging an area, and for displaying an image in real space in response to the user performing a predetermined paste operation on the target area A display control unit that controls the display device to display an image acquired by the copy operation at a position specified by the paste operation.

JP, 2012-3442, A JP, 2013-114467, A

  In the techniques disclosed in Patent Documents 1 and 2 described above, in the display of image information, the space and area to be worked are limited. Then, processing for image information in an area desired by the user, for example, specification of a print area cannot be performed. As a result, it is impossible to perform appropriate image information processing according to the user's request.

  An object of the present invention is to provide an information processing apparatus capable of performing appropriate image information processing according to a user's request.

The information processing apparatus according to the present invention includes a display unit, a recognition unit, a cutout unit, a first control unit, and a second control unit. The display unit is an optical transmission type and displays an electronic image that can be visually recognized by the user. The recognition unit recognizes a predetermined portion of a predetermined object that has passed through the display unit. The cutout unit cuts out an area specified by the predetermined location recognized by the recognition unit as a cutout image. The first control unit performs control so that the cut-out image cut out by the cut-out unit is displayed on the display unit. The second control unit controls to process the cut-out image data. The first control unit controls the display unit to display a transmission state of the cut-out image data when the first control unit transmits the cut-out image data to an external device capable of communicating with the information processing device. Do. The first control unit controls the clipped image so as to move to the external apparatus side according to the transmission state of the clipped image data and to display the clipped image smaller as the user moves away from the user. The second control unit controls the transmission of the cut-out image data to be stopped when the recognition unit recognizes that a shielding object appears between the cut-out image and the external device on the display unit.

  According to such an information processing apparatus, in the area displayed by the display unit, the area designated by the predetermined location is cut out as a cut-out image, and the data of the cut-out image is processed. Then, it is possible to perform processing on image information in an area desired by the user. Therefore, it is possible to perform appropriate image information processing according to the user's request.

It is a block diagram which shows the structure of the information processing system containing the smart glass at the time of applying the information processing apparatus which concerns on one Embodiment of this invention to the smart glass which is a head-mounted terminal device. It is a flowchart which shows the flow of a process in the case of performing the process which specifies an area | region using a smart glass and cuts out an image. It is a figure which shows an example of a cut-out image. FIG. 6 is a diagram illustrating an example of a display mode of data regarding the state of a multifunction peripheral displayed on a lens. It is a figure which shows the transmission state of the data of the cutout image displayed on the lens.

  Embodiments of the present invention will be described below. FIG. 1 is a block diagram showing the configuration of an information processing system including a smart glass when the information processing apparatus according to an embodiment of the present invention is applied to a smart glass which is a head-mounted terminal device.

  Referring to FIG. 1, an information processing system 24 includes a multifunction machine 11 as an image forming apparatus that is an external apparatus for forming an image, and a head-mounted terminal device as an information processing apparatus according to an embodiment of the present invention. Specifically, the smart glass 31 is included. The smart glass 31 is configured to be able to communicate with the multifunction machine 11.

  Next, the configuration of the multifunction machine 11 will be described. The multifunction machine 11 includes a multifunction machine control unit 12, an operation unit 13, a reading unit 14, a printing unit 15, a sheet feeding cassette 16, and a multifunction machine hard disk 17 as a storage unit for storing information of images and settings. , A network interface unit 18 for connecting to a network (not shown), and a multifunction device communication unit 23.

  The multifunction device control unit 12 controls the entire multifunction device 11. The MFP control unit 12 includes a CPU or the like, and includes a main storage memory 19 as a storage unit for temporarily storing data. The operation unit 13 includes a touch panel 21 as a display screen for displaying information. The reading unit 14 reads an image of a document. The printing unit 15 prints an image formed on the basis of an image read by the reading unit 14 or an external device transmitted from an external device via the MFP communication unit 23 or the network, on a sheet.

  The multifunction device communication unit 23 can establish communication with an external device, in this case, for example, the smart glass 31, and can transmit and receive data. The multifunction machine communication unit 23 allows the multifunction machine 11 to receive an image transmitted from the smart glass 31 side. Here, the communication may be wireless communication or wired communication.

  Next, the configuration of the smart glass 31 will be described. The smart glass 31 is a so-called glasses-type head-mounted terminal device. The smart glass 31 includes a smart glass control unit 32, a smart glass communication unit 33, a smart glass storage unit 34 for storing data, a pair of optical transmission type lenses 35 as a display unit, and a user can use the smart glass 31. It includes a pair of arm portions (not shown) corresponding to the portion that is worn by the ear when worn on the head. Further, although not shown, various buttons are provided on the arm portion and the like.

  The smart glass control unit 32 controls the entire smart glass 31. The smart glass control unit 32 is also composed of a CPU or the like, and includes a main memory (not shown) as a storage unit for temporarily storing data. The smart glass communication unit 33 can establish communication with an external device, in this case, for example, the multi-function device 11 to transmit and receive data. About a pair of lens 35, the electronic image which a user can visually recognize is displayed, respectively. This display is controlled by a first control unit 43 described later.

  Next, the configuration of the smart glass control unit 32 provided in the smart glass 31 will be described. The smart glass control unit 32 includes a recognition unit 41, a cutout unit 42, a first control unit 43, and a second control unit 44. The recognition unit 41 recognizes a predetermined portion of a predetermined object that has passed through the lens 35. The cutout unit 42 cuts out an area specified by a predetermined location recognized by the recognition unit 41 as a cutout image. The first control unit 43 performs control so that the cutout image cut out by the cutout unit 42 is displayed on the lens 35. The second control unit 44 performs control so as to process the data of the cutout image. These configurations will be described in detail later.

  Next, a description will be given of a case where a process for cutting out an image by designating an area using the smart glass 31 is performed. FIG. 2 is a flowchart showing a flow of processing in the case of performing processing of specifying an area using the smart glass 31 and cutting out an image.

  Referring to FIG. 2, first, the user wears smart glass 31 on the head. Then, a space or a region that is a work target desired by the user appears on the lens 35 in the user's field of view. Then, for the image visually recognized through the lens 35 of the smart glass 31 by the user, a request for cutting out the image is received (step S11 in FIG. 3, hereinafter, the “step” is omitted). The acceptance of this request is performed, for example, by detecting the pressing of a cutout button (not shown) provided on the smart glass 31.

  Thereafter, the recognition unit 41 recognizes a predetermined portion of a predetermined object that has passed through the lens 35 (S12). In this case, an object which is viewed through the lens 35 and identified as a predetermined object, for example, a user's hand appearing through the lens 35 is a predetermined object, and the user's finger is recognized as a predetermined position. This recognition is performed, for example, by pressing a recognition button (not shown) provided on the smart glass 31 when a newly appearing user's hand appears on an image appearing on the lens 35. .

  Thereafter, a trajectory of an image cut out by the user is drawn with a finger recognized by the recognition unit 41 in the user's hand as a predetermined location. The recognition of the start point and the end point of the trajectory is also performed by detecting the depression of the start point recognition button (not shown) and the end point recognition button provided on the smart glass 31. In this case, it may be detected that the finger trajectory started from the starting point returns to the starting point again.

  Then, the cutout unit 42 cuts out the specified area surrounded by the trajectory as a cutout image (S13). That is, the cutout unit 42 cuts out a region specified by a predetermined location recognized by the recognition unit 41 as a cutout image. In this case, the cut-out image is cut out as a so-called two-dimensional image. At this time, the data of the cutout image may be stored in the smart glass storage unit 34 in response to the detection of pressing of a storage button (not shown) provided on the smart glass 31.

  The clipped image is controlled to be displayed on the lens 35 by the first control unit 43. That is, the first control unit 43 controls to display the cut-out image on the lens 35 (S14).

  Note that the clipped image may be displayed separately from the trajectory as a three-dimensional image. FIG. 3 is a diagram illustrating an example of a cut-out image. FIG. 3 shows a state visually recognized through the lens 35.

Referring to FIG. 3, in a document 51 such as newspaper that is visually recognized through a lens 35, a specified region is surrounded by a locus 53 drawn from the start point to the end point by a user's finger 52. Cut out as The clipped image 54 is indicated by a one-dot chain line in FIG. In this case, the boundary of the cutout image 54 is displayed in a color that can be easily identified, for example, red. The first control unit 43, the cutout image 54 is controlled so as to display the locus 53 of the front of the document 51 on the upper side indicated by the arrow D ₁ with a space to be cut out. Specifically, control is performed so that the cutout image 54 is displayed at a distance from the document 51 by about several centimeters to several tens of centimeters. In this case, the cut-out image 54 appears to float on the document 51 through the lens 35. The clipped image 54 can be grasped by the finger 52 through the lens 35 or moved by grabbing the lens 35 on the display by the first control unit 43. That is, the clipped image 54 is configured to be movable.

  Here, the cutout image 54 may be configured to reflect the change when the original image is changed. That is, when the original 51 is not a printed matter such as newspaper, but is a display device for electronically displaying an image, such as an image of a television or an image of the Internet on a personal computer screen, The images inside may change or change. The data of the contents of the change or change may be received from the display device that can communicate with the smart glass 31, and the first control unit 43 may reflect the change or change in the cutout image 54 and may display it. By doing so, the original image can be reflected in the cutout image 54 in real time, and the temporal change of the original image can be visually recognized through the lens 35 in the cutout image 54.

  Next, the second control unit 44 performs control to process the data of the cutout image 54. In this case, control is performed so that the data of the cut-out image 54 is transmitted to the multifunction device 11 for processing (S15). That is, the data of the cutout image 54 is transmitted to the multi-function device 11 by the second control unit 44 via the smart glass communication unit 33.

  Here, the second control unit 44 may transmit the data of the cutout image 54 to the multifunction device 11 if the flick operation of the cutout image 54 toward the multifunction device 11 is recognized by the recognition unit 41. The multi-function device communication unit 23 receives the data of the cutout image 54 transmitted from the smart glass 31 side (S16). After the reception is completed, the multifunction machine 11 prints on the paper based on the cutout image 54 by the printing unit 15 (S17).

  According to such smart glass 31, in the area displayed by the lens 35, the area designated by the predetermined place is cut out as the cutout image 54, and the data of the cutout image 54 is processed. Then, it is possible to perform processing on image information in an area desired by the user. Therefore, it is possible to perform appropriate image information processing according to the user's request.

  In this case, the first control unit 43 performs control so that the cutout image 54 is displayed on the upper side with an interval from the portion before being cut out. Therefore, the cutout image 54 can be easily recognized visually.

  Further, in this case, when the second control unit 44 recognizes the flick operation of the cutout image 54 toward the multifunction device 11 by the recognition unit 41, the second control unit 44 transmits the data of the cutout image 54 to the multifunction device 11. Therefore, the cut-out image 54 can be transmitted to the multifunction device 11 more easily.

  In the above-described embodiment, the first control unit 43 may acquire data regarding the state of the communicable multifunction device 11 and control the acquired data regarding the state to be displayed on the lens 35.

  FIG. 4 is a diagram illustrating an example of a display mode of data related to the state of the multifunction machine 11 displayed on the lens 35. In FIG. 4, the direction of the flick operation at the time of transmitting the data of the cutout image 54 to each of the multi-function devices 11, 26, 27 is indicated by a two-dot chain line from the viewpoint of easy understanding. Referring to FIG. 4, the first control unit 43 acquires data related to the states of the multifunction machines 11, 26, and 27. Then, the first control unit 43 determines the state of each of the multifunction machines 11, 26 and 27, for example, consumables corresponding to each of the multifunction machines 11, 26 and 27, for example, toner and paper of each of the multifunction machines 11, 26 and 27. Data 56a, 56b, and 56b relating to the remaining amount of are displayed on the lens 35. Specifically, for the data 56a, the information “remaining toner information K good C good M good Y good” and information “remaining paper information A4 many B4 good A3 many” are displayed. Note that “K” indicates a black toner, “C” indicates a cyan toner, “M” indicates a magenta toner, and “Y” indicates a yellow toner. A4, B4, and A3 each indicate the size of the paper. In addition, with regard to the data 56b, information “remaining toner information K good C small M small Y small” and information “remaining paper information A4 small B4 good A3 small” are displayed. Further, for the data 56c, information "remaining toner information K good C good M good Y small" and information "remaining paper information A4 many B4 good A3 little" are displayed. By configuring in this manner, the user can determine the transmission destination of the data of the cut-out image 54 in accordance with the state of the consumables of each of the multifunction machines 11, 26, 27. In this case, it is possible to avoid a situation where printing can not be performed due to a shortage of consumables, or to select MFPs 11, 26, 27 as close to the user as possible within the printable range. The configuration of each of the multi-function devices 26 and 27 is the same as that of the multi-function device 11, and thus the description thereof is omitted.

  In the above embodiment, the first control unit 43 displays the transmission state of the data of the cutout image 54 on the lens 35 when the data of the cutout image 54 is transmitted by the second control unit 44. You may control. In addition, the first control unit 43 controls the cutout image 54 to move toward the multifunction peripheral 11 according to the transmission state of the data of the cutout image 54 and to display the cutout image 54 as being smaller from the user. Good.

  FIG. 5 is a diagram illustrating a transmission state of data of the cutout image 54 displayed on the lens 35. Referring to FIG. 5, a cutout image 54 and three multifunction peripherals 11, 26, and 27 are displayed on the lens 35.

  The data of the cut-out image 54 is transmitted to the multifunction device 11 side by the user's flick operation. Then, a cutout image 57a in which the size of the cutout image 54 is slightly reduced is displayed. Thereafter, as the transmission progresses, the cut-out image 57b having a further reduced size is displayed on the side closer to the multifunction machine 11. With this configuration, the transmission state can be easily recognized visually.

  Here, if the second control unit 44 recognizes that the shielding object appears between the cutout image 54 and the multifunction machine 11 in the lens 35 by the recognition unit 41, the data of the cutout image 54 by the second control unit 44 Control may be performed so as to cancel the transmission of. For example, in the transmission state of the cutout image 57a shown in FIG. 5, it is assumed that a shield, for example, a user's hand is placed in the area 58 between the cutout image 57a and the multifunction machine 11, and the shield is displayed. Then, the user's hand is recognized by the recognition unit 41, and the second control unit 44 stops transmission of the cutout image 57a. By doing so, it is possible to easily perform an operation of stopping the transmission after the transmission of the data of the cutout image 54.

  A plurality of multifunction machines 11, 26, 27 are provided, and the first control unit 43 recognizes that the plurality of multifunction machines 11, 26, 27 are present in the direction of the flick operation. Control to display on the lens 35 a screen asking whether to send to the multifunction devices 11, 26, 27 and the second control unit 44 cuts out the image 54 to the multifunction devices 11, 26, 27 specified by the query screen. It may be controlled to transmit data. By doing so, the data of the cut-out image 54 can be transmitted to the multifunction peripherals 11, 26, and 27 desired by the user more reliably. That is, in FIG. 4, arrows shown by two-dot chain lines point in different directions, but such control may be performed when these directions approach or overlap.

  In the above embodiment, although the recognition unit 41 recognizes the user's finger 52 as a predetermined part of a predetermined object, the present invention is not limited to this. For example, a pen or a rod-like member is recognized as a predetermined object You may do it. That is, the predetermined part of the predetermined object recognized by the recognition unit 41 may be configured to be at least one of the end of the rod-shaped member and the user's finger. In this case, for example, using a pen with a lit pen tip, the pen tip is turned on while drawing a locus, and otherwise the pen tip is turned off to recognize the start point and the end point. It is good also as composition like this.

  In the above embodiment, the second control unit 44 causes the printing unit 15 to print on a sheet based on the data of the cut-out image 54. However, the present invention is not limited to this. Other processing, for example, processing may be performed so that the data of the cut-out image 54 is stored in the multifunction device hard disk 17 provided in the multifunction device 11. Here, the first control unit 43 may control to display the capacity of the MFP hard disk 17 and the state of the box name as the storage destination as the states of the MFPs 11, 26 and 27 described above. Note that the data of the cutout image 54 stored in the multi-function device hard disk 17 may be displayed on the touch panel 21 so that some processing or editing can be performed.

  In the above-described embodiment, a configuration in which the clipped image 54 can be added or written can be adopted. That is, in the setting on the smart glass 31 side, the user's finger 52 is set as a writing pen and written on the cutout image 54. Specifically, an underline is drawn or a marker is drawn on the text image. The written image can be visually recognized on the lens 35 of the smart glass 31 by the first control unit 43. By configuring in this way, it is possible to perform appropriate image processing according to the user's request.

  Further, in the above embodiment, a plurality of portions cut out by the cut-out unit 42 may be configured, and the plurality of cut-out images cut out may be composited in the virtual space displayed by the first display unit 43. .

  In the above embodiment, the information processing apparatus according to the present invention is applied to the smart glass 31, which is a head-mounted terminal apparatus. However, the present invention is not limited to this. For example, a display function for displaying an image It is good also as applying to the smart phone which has, and another portable terminal device. In this case, of course, it may not be a head-mounted type.

  It should be understood that the embodiments disclosed herein are illustrative in all respects and are not restrictive in any aspect. The scope of the present invention is defined by the scope of the claims, rather than the description above, and is intended to include any modifications within the scope and meaning equivalent to the scope of the claims.

  The information processing apparatus according to the present invention is particularly effectively used when processing of appropriate image information according to a user's request is required.

  11, 26 and 27 MFPs, 12 MFP control parts, 13 operation parts, 14 reading parts, 15 printing parts, 16 paper feed cassettes, 17 MFP hard disks, 18 network interface parts, 19 main memory, 21 touch panels, 23 MFP communication unit, 24 information processing system, 31 smart glasses, 32 smart glass control units, 33 smart glass communication units, 34 smart glass storage units, 35 lenses, 41 recognition units, 42 cutout units, 43 first control units, 44 Second control unit, 51 document, 52 fingers, 53 locus, 54, 57a, 57b cut image, 56a, 56b, 56c data, 58 area.

Claims (6)

An information processing apparatus,
An optically transmissive display that displays an electronic image visible to the user;
A recognition unit for recognizing a predetermined part of a predetermined object that has passed through the display unit;
A cutout unit that cuts out an area designated by the predetermined location recognized by the recognition unit as a cutout image;
A first control unit that controls to display the cut-out image cut out by the cut-out unit on the display unit;
A second control unit that controls to process the data of the cut-out image ,
The first control unit transmits the cut-out image data to the display unit when the first control unit is transmitting the cut-out image data to an external device capable of communicating with the information processing apparatus. Control to display the status,
The first control unit controls the clipped image to move toward the external device according to a transmission state of the clipped image data, and to display the clipped image smaller as the user moves away from the user.
The second control unit controls the transmission of the cut-out image data to be stopped when the recognizing unit recognizes that a shielding object appears between the cut-out image and the external device on the display unit. , Information processing device.   The information processing apparatus according to claim 1, wherein the first control unit controls the clipped image to be displayed on the upper side with a space from the portion before being clipped. The second control unit, if recognizing the flick operation toward the external apparatus of the cutout image by the recognition unit, and controls to transmit the data of the clipped image to the external device, according to claim 1 or 2 The information processing apparatus according to claim 1. A plurality of the external devices are provided,
If the first control unit recognizes that there are a plurality of the external devices in the direction of the flick operation, the first control unit displays an inquiry screen asking which of the external devices is to be transmitted to the display unit. Control
The information processing apparatus according to claim 3 , wherein the second control unit performs control so that the data of the cut-out image is transmitted to the external apparatus specified by the inquiry screen. The external device includes a printing unit that prints an image on paper based on image data,
It said second control unit controls to print on paper on the basis of data of the clipped image by the printing unit, the information processing apparatus according to any one of claims 1-4. Said predetermined portion of the predetermined object to be recognized by the recognition unit, at least at either of the ends and the user's finger bar-shaped member, information according to any one of claims 1 to 5 Processing unit.

Images