JP7216610B2 - DATA PROCESSING APPARATUS, DATA PROCESSING METHOD, AND PROGRAM - Google Patents

Description

TECHNICAL FIELD The present invention relates to a data processing device and related technology, and more particularly to technology for executing rendering processing based on a data set detected by a detection device.

Acquire a data set (fish school data, etc.) detected by a detection device (transceiver device, etc.) for a target (fish school, etc.), execute rendering processing based on the data set, and display the image generated by the rendering processing on the screen. There is technology to display within. Further, different images (specifically, navigation images, radar images, and underwater detection images) based on different data are displayed in a plurality of rectangular areas (divided areas) provided in the display screen. There is a technique for receiving an instruction such as movement (scrolling) or scaling (enlargement/reduction) by a touch gesture in each rectangular area (see, for example, Patent Document 1). In this technique, it is determined to which divided area the operation position of the touch gesture belongs, and the target area (target image) of the touch gesture is specified based on the determination result. Then, a scrolling process, an enlarging/reducing process (variable magnification process), or the like is performed within the specified divided area (area corresponding to the touch gesture).

U.S. Patent Application Publication No. 2015/0035772

In the related art as described above, the target area of the touch gesture is determined according to which of the plurality of divided areas (rectangular areas) the operation position of the touch gesture belongs to.

By the way, there is a demand for improving the degree of freedom in arranging a plurality of regions. When a plurality of areas are arranged according to such a request, for example, a plurality of adjacent rectangular areas may be arranged so as to partially overlap.

However, it is not necessarily easy to appropriately determine the target area of the touch gesture when a plurality of rectangular areas are partially overlapped.

Accordingly, it is an object of the present invention to provide a technology capable of more appropriately determining a gesture target area from among a plurality of areas.

(1) In order to solve the above problems, a data processing device according to one aspect of the present invention includes a data acquisition unit that acquires a data set detected by a detection device with respect to a target, and a rendering process based on the data set. and a rendering unit that generates a plurality of views arranged in a screen, each of the plurality of views including a plurality of pixels, each pixel included in the plurality of views being displayed on the screen. A plurality of pieces of information including first information to be displayed and second information indicating a view to which each pixel belongs among the plurality of views are associated with each other.

(2) The data processing device includes a user interface unit that receives an operation by a user on the plurality of views, and acquires the second information associated with the target pixel of the operation by the user, and performs the operation. A view identification unit that identifies an operation target view that is a target view based on the second information of the target pixel, a gesture detection unit that detects a gesture related to the operation as an instruction gesture, and the instruction gesture is a view correction instruction. In some cases, the method may further include a view modification unit that modifies the operation target view based on the pointing gesture.

(3) The plurality of views may include at least two of a perspective view, a side view and a top view.

(4) The plurality of views include a perspective view, and the view modification unit determines that the operation target view is the perspective view, and that the amount of movement of the instruction gesture in the horizontal direction within the screen is within the screen. is greater than the amount of movement of the pointing gesture in the vertical direction of the first A view image obtained by rotating the viewpoint of the perspective view around an axis may be generated as a new perspective view.

(5) The plurality of views include a perspective view, and the view modification unit determines that the operation target view is the perspective view, and the amount of movement of the instruction gesture in the horizontal direction within the screen is within the screen. a second axis in the three-dimensional space for the dataset that corresponds to the horizontal direction in the screen in the perspective view, if less than the movement of the pointing gesture in the vertical direction of A view image obtained by rotating the viewpoint of the perspective view around an axis may be generated as a new perspective view.

(6) The plurality of views include a side view, and the view modification unit determines that the operation target view is the side view, and that the amount of movement of the instruction gesture in the horizontal direction within the screen is within the screen. is greater than the amount of movement of the pointing gesture in the vertical direction of the first A view image obtained by rotating the viewpoint of the side view around an axis may be generated as a new side view.

(7) The plurality of views include a side view, and the view modification unit determines that the operation target view is the side view, and the amount of movement of the instruction gesture in the horizontal direction within the screen is within the screen. is less than the amount of movement of the pointing gesture in the vertical direction, the side-view viewpoint may not be rotated.

(8) The plurality of views include a top view, and the view modification unit determines that the view to be operated is the top view, and an instruction to move in the horizontal direction or the vertical direction within the screen is given based on the instruction gesture. If given, rotate the top-view point of view about a first axis in the three-dimensional space for the dataset that corresponds to a direction perpendicular to the screen in the top-view. A new view image may be generated as a new top view.

(9) The view modification unit may modify at least one view other than the operation target view among the plurality of views when the view modification instruction for the operation target view is given.

(10) The plurality of views include a first view that is one of a perspective view, a side view, and a top view, and one of the perspective view, the side view, and the top view other than the first view. and a second view that is a view, wherein the view modification unit is configured such that the operation target view is the first view, and the view modification instruction is given to the first view, and the data set If the first view viewpoint is rotated about a first axis in the three-dimensional space of to update the first view, then rotate the second view viewpoint about the first axis may also update the second view.

(11) The data processing device includes a transmitting unit that transmits a transmitted wave, a receiving unit that receives a received wave including the reflection of the transmitted wave from the target, and generates a received signal based on the received wave, a data generator that generates the data set based on the received signal, wherein the received wave is received from a three-dimensional space extending outward from the receiver.

(12) In order to solve the above problems, a data processing method according to one aspect of the present invention obtains a data set detected by a detection device with respect to a target, executes rendering processing based on the data set, wherein each of said plurality of views comprises a plurality of pixels, each pixel included in said plurality of views comprising a first piece of information displayed on said screen. and second information indicating the view to which each pixel belongs among the plurality of views.

(13) In order to solve the above problems, a program according to an aspect of the present invention provides a computer with a) acquiring a dataset detected by a detection device with respect to a target; b) rendering based on the dataset; and performing processing to generate a plurality of views in a screen, each of the plurality of views comprising a plurality of pixels, each pixel included in the plurality of views is associated with a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views.

According to the present invention, each pixel included in a plurality of views is associated with a plurality of pieces of information including the second information indicating the view to which the pixel belongs among the plurality of views. 2, it is possible to easily obtain information about the view to which each pixel belongs. As a result, it is possible to more appropriately determine the gesture target area from among the plurality of areas respectively corresponding to the plurality of views.

1 is a block diagram showing the configuration of an underwater detection device (data processing device); FIG. It is a block diagram which shows the functional structure of an information processing apparatus. FIG. 2 is a diagram schematically showing a transmission space in which transmission waves are transmitted by a transducer and a plurality of reception spaces in which reception waves are received by the transducer; It is a figure which shows the display screen displayed on a display part. 4 is a flow chart showing the operation of the underwater detection device; FIG. 10 is a diagram showing a movement vector of a drag operation; FIG. FIG. 10 is a diagram showing a state in which a rightward drag operation is performed on a perspective view; FIG. 11 is a diagram showing a perspective view after change; FIG. 10 is a diagram showing how a downward drag operation is performed on a perspective view; FIG. 11 is a diagram showing a perspective view after change; FIG. 11 is a diagram showing a state in which a rightward drag operation is performed on a side view; It is a figure which shows the side view after a change. FIG. 10 is a diagram showing a state in which a downward drag operation is performed on the side view; FIG. 10 is a diagram showing how a drag operation is performed on the top view; FIG. 10 is a diagram showing how a drag operation is performed on the top view; FIG. 10 is a diagram showing how a drag operation is performed on the top view; It is a figure which shows the top view after a change. FIG. 10 is a diagram showing a state in which a plurality of views are linked and changed; It is a figure which shows a mode that an isodepths line is dragged. FIG. 11 shows a perspective view after the depth contours have been moved; FIG. 10 is a diagram showing the display screen after the top view has been moved; FIG. 10 is a diagram showing a display screen after the fluoroscopic view is enlarged; FIG. 10 is a diagram showing a display screen after each view has been moved and enlarged; FIG. 2 is a conceptual diagram showing how a certain pixel within a display screen is associated with first information and second information;

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings.

<1. First Embodiment>
<1-1. Overall Configuration of Underwater Detector>
FIG. 1 is a block diagram showing the configuration of a data processing device (here, an underwater detection device 1) according to the present invention. The underwater detection device 1 of this embodiment is used, for example, in a vessel such as a fishing boat.

An underwater detection device 1 (also called an environment detection device) includes a scanning sonar 10 and an information processing device 20, as shown in FIG. The underwater detection device 1 has a configuration in which, for example, an information processing device 20 is externally attached to a generally known scanning sonar 10 . The underwater detection device 1 may have a configuration in which the information processing device 20 is mounted on the scanning sonar 10 instead of being externally attached to the scanning sonar 10 . A display unit 31 configured as a display device such as a display is externally attached to the information processing device 20 . Further, the underwater detection device 1 is externally provided with an operation input section 33 configured as a pointing device such as a mouse. The display section 31 and the operation input section 33 function as a user interface section.

A scanning sonar 10 includes a transducer 2 and a transmitter/receiver 3 .

<1-2. Configuration of transducer>
The transducer 2 has a function of transmitting and receiving ultrasonic waves, and is attached to the bottom of the ship S. For example, as an example, the transducer 2 is formed in a substantially spherical shape.

Specifically, the transducer 2 has a substantially spherical housing and ultrasonic transducers (not shown) as a plurality of transducer elements attached to the outer peripheral surface of the housing. The ultrasonic transducer transmits ultrasonic waves to an underwater transmission space as transmitted waves, receives received waves as reflected waves including the reflection of the transmitted waves from targets in the water, and converts the received waves into electrical signals. By doing so, a received signal is generated from the received wave and output to the transmitter/receiver 3 . That is, the wave transducer 2 is configured as a transmission section (also referred to as a transmission transducer) that transmits transmission waves underwater, receives reception waves including reflection of the transmission waves at targets in water, and receives received waves. It is configured as a receiving section (also called a receiving transducer) that generates a received signal from waves. A school of fish or the like is an underwater target on which the transmission wave transmitted from the transducer 2 is reflected.

In this embodiment, the transducer 2 has a spherical case, but the shape is not particularly limited, and other shapes such as a substantially cylindrical shape may be used. good too. When the housing of the transducer 2 has a substantially cylindrical shape, the transducer 2 is arranged so that its axial direction is along the vertical direction and its radial direction is along the horizontal direction.

FIG. 3 is a diagram schematically showing a transmission space TS in which transmission waves are transmitted by the transducer 2 and a plurality of reception spaces RS in which reception waves are received by the transducer 2. As shown in FIG. The transmission waves transmitted from the transducer 2 mounted on the ship S are simultaneously transmitted from the transducer 2 in all directions in the water around the ship S, forming, for example, a hemispherical transmission beam. be done. When a hemispherical transmission beam is formed, the transmission space TS through which the transmission wave is transmitted is configured as a hemispherical space. The shape of the transmission beam is not limited to a hemispherical shape, and can be formed in various different shapes depending on the shape of the transducer 2 or the amplitude and phase of the electrical signal input to each transducer element of the transducer 2. be.

Further, the transducer 2 receives received waves from a three-dimensional space extending outward from the transducer 2 . Specifically, after transmitting the transmission beam, the transducer 2 simultaneously forms a plurality of reception beams that scan in the circumferential direction (in the direction of the azimuth angle θ indicated by the arrow in FIG. 3) in the transmission space TS. . That is, all reception beams are formed at one reception timing by the transducer 2 . Then, received waves reflected by targets such as schools of fish in water are arranged in each of a plurality of receiving spaces RS (that is, , in each space in which the receive beams are formed).

<1-3. Configuration of Transmitter/Receiver>
The transceiver 3 includes a transmission/reception switching section 3a, a transmission circuit section 6, and a reception circuit section 7 (see FIG. 1).

The transmission/reception switching unit 3 a is for switching between transmission and reception of a signal to the transducer 2 . Specifically, when transmitting a drive signal for driving the transducer 2 to the transducer 2, the transmission/reception switching unit 3a outputs the drive signal output by the transmission circuit unit 6 to the transducer 2. . On the other hand, when receiving a reception signal from the transducer 2 , the transmission/reception switching unit 3 a outputs the reception signal received from the transducer 2 to the reception circuit unit 7 .

The transmission circuit section 6 generates a drive signal that is the basis of transmission waves transmitted from the transducer 2 . More specifically, the transmission circuit section 6 has a transmission circuit (not shown) provided corresponding to each ultrasonic transducer, and each transmission circuit generates a drive signal.

The receiving circuit section 7 has an analog section 7a and an A/D conversion section 7b. The analog unit 7a and the A/D conversion unit 7b are reception circuits provided corresponding to each ultrasonic transducer, and are reception circuits (not shown) that process reception signals generated from received reception waves. is configured with The analog section 7a amplifies the received signal as an electrical signal generated from the received wave by the transducer 2 and outputs the received signal, and limits the band of the received signal to remove unnecessary frequency components. The A/D conversion section 7b converts the received signal amplified by the analog section 7a into a received signal as a digital signal. Then, the receiving circuit section 7 outputs the received signal converted into a digital signal by the A/D converting section 7 b to the information processing device 20 .

<1-4. Configuration of Display Unit>
The display unit 31 is configured as a display device such as a display. The display unit 31 displays a video (image) corresponding to the video signal output from the information processing device 20 on the display screen. The display unit 31 displays, for example, the underwater state below the ship S three-dimensionally. A plurality of views (view images) 51, 52, and 53 are displayed on the display screen of the display unit 31 (see FIG. 4, etc.), as will be described in detail later. Note that the user of the underwater detection device 1 can see the display screen and guess the state of the sea below the ship S (for example, the presence or absence and position of structures such as schools of fish, undulations of the seabed, and artificial reefs). can be done.

<1-5. Configuration of Information Processing Device>
FIG. 2 is a block diagram showing the functional configuration of the information processing device 20. As shown in FIG. As shown in FIGS. 1 and 2, the information processing device 20 processes the received signal output from the receiving circuit section 7, generates a target echo signal, and causes the display section 31 to display the target echo signal. processing to generate an echo video signal for the purpose. Since the information processing device 20 has a rendering function, it is also called a rendering device or a display control device.

The information processing device 20 includes a data generation unit 21, a data acquisition unit 22, a rendering unit 23, a view identification unit 24, a gesture detection unit 25, a view correction unit 26, and the like.

The information processing device 20 is a device connected to the transmitter/receiver 3 of the scanning sonar 10 by a cable or the like, and configured as, for example, a PC (personal computer). The information processing device 20 includes a hardware processor (eg, CPU (Central Processing Unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array), etc.), various memories (volatile memory and non-volatile memory), etc. device.

Various processing units are realized by executing a predetermined software program (hereinafter simply referred to as a program) stored in a non-volatile memory in the hardware processor. The program (specifically, the program module group) may be recorded in a portable recording medium such as a USB memory, read out from the recording medium, and installed in the information processing apparatus 20 . Alternatively, the program may be downloaded via a network and installed in the information processing device 20 .

Various processing units including the data generation unit 21, the data acquisition unit 22, the rendering unit 23, the view identification unit 24, the gesture detection unit 25, and the view correction unit 26 are executed by executing the program in the hardware processor. Realized.

The data generator 21 is a processor that generates a data set (echo data, etc.) based on the received signal generated by the transmitter/receiver 3 . The data generation unit 21 generates the data set by performing beamforming processing on each of the plurality of reception spatial RSs based on the received signal received from the transceiver 3 . The data set is a group of data detected by the detection device with respect to a target (school of fish, etc.). For example, the data set includes data on echo intensity (intensity of reflected waves from a school of fish, etc.) at each three-dimensional position (X, Y, Z) in the underwater environment. The data set is temporarily stored in a volatile memory (or non-volatile memory) or the like.

The data acquisition unit 22 is a processing unit that acquires a data set (fish school data, etc.) detected by a detection device (scanning sonar 10, etc.) with respect to a target (fish school, etc.). The data acquisition unit 22 acquires, for example, echo data detected by the transducer 2 and the transmitter/receiver 3 with respect to a target (such as a school of fish) and stored in the memory by extracting from the memory.

The rendering unit 23 is a processing unit that executes rendering processing based on data sets. The rendering unit 23 generates a plurality of views (also referred to as view images) (described later) arranged within the display screen of the display unit 31 .

The view identification unit 24 acquires second information F2 (described later) associated with the target pixel of the user's operation, and identifies the operation target view (the target view of the operation) based on the second information F2. It is a processing unit that In other words, the view identifying unit 24 is a processing unit that identifies a gesture target area from among a plurality of areas respectively corresponding to a plurality of view images. By using the second information F2, it is possible to easily identify the view to be operated.

The gesture detection unit 25 is a processing unit that detects a gesture accompanying a user operation as an instruction gesture (operation instruction information). A user can operate any one of the plurality of views through the user interface unit. A "gesture" by the user is accepted by a user interface unit (eg, a pointing device such as a mouse).

A "gesture" is accepted, for example, through a user's mouse operation. In other words, various mouse operations (clicking, dragging, rotating the mouse wheel, etc.) are accepted as "gestures." Note that the "gesture" is not limited to this, and may be received by another user interface unit such as a touch panel (touch screen). For example, a direct touch operation (drag operation, pinch-in operation, pinch-out operation, etc.) on the screen may be accepted as a "gesture". In this first embodiment, a manner in which gestures are accepted using a mouse will be mainly described. In the second embodiment, a manner in which a gesture is accepted using a touch panel will be mainly described.

The view correction unit 26 is a processing unit that cooperates with the rendering unit 23 to correct the view. When the instruction gesture is a view correction instruction, the view correction unit 26 corrects the operation target view (operation target view image) based on the view correction instruction.

<1-6. Operation>
<Initial operation>
FIG. 5 is a flow chart showing the operation of the underwater detection device 1 (mainly the information processing device 20). The operation of the underwater detection device 1 will be described below with reference to FIG. 5 and the like.

First, in step S11, the information processing device 20 (specifically, the data acquisition unit 22) of the underwater detection device 1 acquires a data set detected from the underwater environment regarding an underwater target (such as a school of fish). Specifically, echo data detected by the scanning sonar 10 or the like (a group of echo data obtained from a hemispherical environmental space extending below the ship S) is acquired as the data set.

Then, in step S<b>12 , the information processing device 20 (more specifically, the rendering unit 23 ) executes rendering processing based on the data set, and generates a plurality of views 50 arranged within the display screen 200 of the display unit 31 . . Then, the information processing apparatus 20 arranges each of the plurality of views at each initial position within the display screen 200 .

<Multiple views>
FIG. 4 is a diagram showing a display screen 200 displayed on the display unit 31. As shown in FIG. Display screen 200 displays multiple views (also referred to as view images) based on the same data set.

As shown in FIG. 4, the multiple views 50 here include three views, a perspective view 51 , a side view 52 and a top view 53 . However, it is not limited to this, and for example, only any two of the perspective view 51 , the side view 52 and the top view 53 may be rendered as the multiple views 50 . Alternatively, at least two (or at least one) of perspective view 51 , side view 52 and top view 53 and other types of views may be included in multiple views 50 . Note that the "perspective view" is also called a perspective view, a 3D view, a perspective image, a perspective image, or the like, a side view is also called a side view, a side view image, or the like, and a top view is a plan view or a top view. , a plan view image, a top view image, or the like.

Each view has a hemispherical (also referred to as bowl-shaped or lower hemispherical) data set display space as a display object.

A perspective view (3D view) 51 is a view image obtained by stereoscopically (three-dimensionally) rendering a data set display space (three-dimensional space for displaying a data set) using a perspective drawing method. The see-through view 51 here has, so to speak, the outer shape of the bowl as viewed obliquely from above (upper front side) in the display screen 200 . Specifically, the perspective view 51 is one of two ellipses (two ellipses having the same axial length in the major axis direction (horizontal direction) and different axial lengths in the minor axis direction (vertical direction)). It has a shape in which an upper half and the other lower half are combined vertically. By visually recognizing the stereoscopically drawn perspective view (3D view) 51, the user can easily grasp the overall image.

A side view 52 is a view image of the data set display space viewed from the side. The side view 52 has a semicircular shape (lower semicircular shape) in the display screen 200 here.

A top view 53 is a view image of the data set display space viewed from directly above. The top view 53 has a circular shape in the display screen 200 here.

<Data structure>
Each of the multiple views 51, 52, 53 includes multiple pixels. Further, as shown in the conceptual diagram of FIG. 24, each pixel 68 included in a plurality of views is associated with the first information F1 displayed on the display screen 200 and the destination of each pixel among the plurality of views. Each is associated with a plurality of pieces of information including second information F2 indicating a view. 24, a certain pixel 68 within the display screen 200 (more specifically, within the perspective view 51) is associated with a plurality of pieces of information (including the first information F1 and the second information F2). FIG. 2 is a diagram conceptually showing a state in which Similarly, for each other pixel in the display screen 200 (other pixels in perspective view 51, each pixel in side view 52 and top view 53, etc.), first information F1 and second information Information F2 and the like are associated with each other.

As described above, the data set comprises data for each echo intensity (intensity of reflected waves from a school of fish, etc.) at multiple three-dimensional locations (X, Y, Z) within the underwater environment. In other words, the data set includes the three-dimensional position (X, Y, Z) of each target and the signal strength of the received signal from each target at that three-dimensional position.

In each of the views 51, 52, 53, information such as echo intensity at each three-dimensional position is distinguished by color and displayed. Specifically, color information such as red, yellow, and blue (echo intensity is divided into a plurality of stages and information in which the plurality of stages are distinguished by color) is given. For example, echo information ( color information according to the echo intensity) is given as the first information F1 for each pixel in the display screen 200 (two-dimensional plane). Alternatively, the echo information of the three-dimensional position having the highest level of echo intensity on a predetermined line of sight corresponding to each pixel of each view and the frontmost three-dimensional position when viewed from a predetermined viewpoint of each view. may be given as the first information F1 of each pixel in the display screen (two-dimensional plane).

In each view 51, 52, 53 of the display screen 200 of FIG. 4, among the pixel groups showing the echo data of each fish school 71, 72, the red pixel group (indicating having the highest signal intensity level) pixel group) are shown with diagonal hatching. In addition, the yellow pixel group (the pixel group having the next highest signal intensity level after the red pixel group) is indicated by sand hatching.

Thus, the first information F1 is associated with each pixel in each view. Note that the first information F1 may be color information based on the echo intensity, or may be an echo intensity step value, an echo intensity value (detected value) itself, or the like.

Second information F2 is also associated with each pixel in each view. The second information F2 of each pixel is information indicating the view to which each pixel belongs among the plurality of views 50 (51, 52, 53). The second information F2 of each pixel is information indicating to which of the plurality of views 51, 52, 53 each pixel belongs, and is also called view identification information.

For example, each pixel in the perspective view 51 is associated with the fact that each pixel belongs to the perspective view 51 as the second information F2. Similarly, each pixel in the side view 52 is associated with second information F2 indicating that each pixel belongs to the side view 52 . Similarly, each pixel in the top view 53 is associated with second information F2 indicating that each pixel belongs to the top view 53 .

Note that here, among the pixels in the display screen 200, pixels that do not belong to any of the plurality of views 51, 52, and 53 are not associated with the second information F2 itself. However, the present invention is not limited to this, and among the pixels of the display screen 200, pixels that do not belong to any of the views 51, 52, and 53 are indicated to that effect (that they do not belong to any of the views 51, 52, and 53). The indicating information (“0” or the like) may be associated as the second information F2.

Also, when two adjacent views overlap each other (apparently), the fact that each pixel in the overlapping area belongs to the upper (upper layer) view is recorded as the second information F2. For example, as shown in FIG. 23 (described later), each pixel in the overlapping portion between the side view 52 and the top view 53 (the area near the left end of the top view 53 and covering the vicinity of the upper right end of the side view 52) belongs to the upper top view 53 is recorded as the second information F2 of each pixel.

<Overview of changes in each view according to user operations>
In the next step S13, the information processing device 20 (more specifically, the gesture detection unit 25) determines whether or not a gesture associated with user operation has been detected. Here, it is assumed that a gesture using a mouse (drag operation using a mouse, etc.) is detected as the gesture.

If the gesture is not detected in step S13, the process proceeds to step S17 to determine whether or not to terminate. In step S17, it is determined whether or not a condition for temporarily ending the processing of FIG. 5, such as the arrival of the acquisition timing of the next data set (next echo data, etc.), has been met. If it is determined that the condition is not satisfied yet, the process returns to step S13. On the other hand, if it is determined that the condition is met, the process of FIG. 5 is once terminated. After that, the processing in FIG. 5 is resumed when the next data set acquisition timing arrives.

On the other hand, if a gesture is detected in step S13, the process proceeds to step S14.

In step S14, in response to the reception of the view correction instruction, first, the operation target view for the gesture is specified. More specifically, the information processing device 20 (view specifying unit 24) acquires the second information F2 associated with the pixel at the operation start position of the drag operation. Then, the operation target view is specified based on the acquired second information F2. Further, in step S<b>15 , the instruction content (correction instruction content) of the instruction gesture for the operation target view is specified, and the view correction unit 26 executes viewpoint change processing (viewpoint rotation processing) based on the instruction content. More specifically, when it is determined (by the gesture detection unit 25 and the view correction unit 26) that the user's gesture (instruction gesture) is a view correction instruction, the viewpoint after change according to the view correction instruction is determined. be done. Further, in step S16, rendering processing based on the changed viewpoint (rendering processing of objects related to the data set) is performed by the rendering unit 23 and the view correction unit 26, and the operation target view is corrected.

<Change of Viewpoint of Perspective View 51>
When the user performs a mouse drag operation (instruction gesture) on the display screen 200 while the mouse cursor 65 is present on the perspective view 51 (see FIGS. 7 and 9), the following processing is executed. be.

In this case, first, the second information F2 associated with the pixel at the operation start position of the drag operation is obtained, and the pixel is associated with the perspective view 51 based on the second information F2. is determined. Based on this determination result, the view specifying unit 24 further determines that the view to be operated is the perspective view 51 (step S14).

Next, by comparing the direction component values ΔU and ΔV (see FIG. 6) of the drag operation (instruction gesture), it is determined whether the horizontal direction (U direction) or vertical direction (V direction) in the screen 200 is determined. is given (step S15). In FIG. 6, the movement vector of the drag operation is indicated by a white arrow, and the horizontal component ΔU (the amount of movement of the mouse in the horizontal direction) and the vertical component ΔV (the amount of movement of the mouse) of the movement vector (white arrow) ) are shown.

Then, the instruction content of the instruction gesture is identified according to the comparison result between the instruction movement amount ΔU in the horizontal direction within the screen 200 and the instruction movement amount ΔV in the vertical direction within the screen 200 (step S15). According to this, the main direction of the drag operation, including the case where the drag operation is performed in an oblique direction within the screen, and thus the instruction content of the instruction gesture are specified.

Specifically, when the instruction movement amount ΔU is larger than the instruction movement amount ΔV, the “rotation instruction about the vertical axis AX1” is specified as the instruction content of the instruction gesture for the operation target view 51 . In other words, an instruction to change the azimuth angle with respect to the viewpoint is specified as the instruction content. The vertical axis AX1 is an axis extending vertically from the ship (own ship) S to the seabed in a three-dimensional space (real space).

On the other hand, when the instruction movement amount ΔU is smaller than the instruction movement amount ΔV, the “rotation instruction about the horizontal axis AX2” is specified as the instruction content of the instruction gesture for the operation target view 51 . In other words, an instruction to change the elevation angle with respect to the viewpoint is specified as the instruction content.

Thus, the rotation direction of the viewpoint is determined. Specifically, the rotation direction of the viewpoint is changed according to the magnitude relationship between the instruction movement amount ΔU and the instruction movement amount ΔV.

Further, in step S15, the rotation amount (movement amount) of the viewpoint is determined according to the movement amount of the drag operation.

When the instruction content of the instruction gesture (rotation direction (moving direction) and amount of rotation of the viewpoint) is determined in this manner, rendering processing based on the changed viewpoint corresponding to the instruction content is executed in the next step S16. and the view is modified. Specifically, a new view image from the changed viewpoint is generated, and the perspective view is updated based on the new perspective view.

More specifically, when the instruction movement amount ΔU is larger than the instruction movement amount ΔV, it corresponds to the vertical axis (Z-axis) AX1 in the three-dimensional space and the vertical direction in the screen 200 in the perspective view 51. A view image obtained by rotating the viewpoint of the perspective view about the vertical axis AX1 is generated as a new perspective view.

For example, as shown in FIG. 7, when the user performs a horizontal rightward drag operation (instruction gesture) (within the display screen 200) in a state where the mouse cursor 65 exists on the perspective view 51, the information processing apparatus 20 determines that the user has given a rotation operation instruction for the perspective view 51 about the vertical axis (Z-axis) in the real space (and the vertical axis (V-axis) on the screen) AX1. Then, as shown in FIG. 8, instead of the perspective view 51 before change, a perspective view 51 after change (a new perspective view) is displayed. FIG. 8 shows a new perspective view with its viewpoint rotated around the vertical axis AX1 (about 90 degrees clockwise). The new perspective view 51 is also expressed as an image obtained by rotating the original view image around the vertical axis AX1 (about 90 degrees counterclockwise). Note that when a leftward drag operation is performed, the perspective view 51 rotated in the opposite direction around AX1 is displayed.

On the other hand, when the instruction movement amount ΔU is smaller than the instruction movement amount ΔV, the perspective view is rotated around the horizontal axis AX2 in the three-dimensional space and corresponding to the horizontal direction in the screen 200 in the perspective view 51 . A view image obtained by rotating the viewpoint of is generated as a new perspective view.

For example, as shown in FIG. 9, when the user performs a vertical downward drag operation (within the display screen 200) in a state where the mouse cursor 65 exists on the perspective view 51, the information processing apparatus 20 performs a perspective Regarding the view 51, it is determined that the user has given a rotational operation instruction about the horizontal axis in the real space (and the horizontal axis (U-axis) on the screen) AX2. Then, as shown in FIG. 10, instead of the perspective view 51 before change, a perspective view 51 after change (a new perspective view in which the viewpoint is rotated around the horizontal axis AX2) is displayed. As can be seen by comparing FIG. 10 and FIG. 9, in FIG. 10 a perspective view is displayed from a viewpoint with a greater elevation angle. Note that when an upward drag operation is performed, the perspective view 51 rotated in the opposite direction around AX2 is displayed.

<Change of Viewpoint of Side View 52>
Also, as shown in FIG. 11 (or FIG. 13), when the user performs a drag operation (drag operation using a mouse) from the state where the mouse cursor 65 exists on the side view 52, the following processing is performed. executed.

In this case, first, it is determined based on the second information F2 of the pixel that the pixel at the operation start position of the drag operation is associated with the side view 52, and the view specifying unit 24 selects the operation target view. is the side view 52 (step S14).

Next, the instruction content of the instruction gesture is specified according to the comparison result between the instruction movement amount ΔU and the instruction movement amount ΔV (see FIG. 6) (step S15), and the view is displayed as necessary based on the instruction content of the instruction gesture. is corrected (step S16). Specifically, whether or not to modify the view is changed according to the magnitude relationship between the instruction movement amount ΔU and the instruction movement amount ΔV.

Specifically, when the instruction movement amount ΔU is larger than the instruction movement amount ΔV (see FIG. 11), “rotation instruction about the vertical axis AX1” is specified as the instruction content of the instruction gesture for the operation target view 52. . Then, a view image obtained by rotating the viewpoint of the side view 52 about the vertical axis AX1 is generated as a new side view 52, and the new side view 52 is displayed on the screen 200 instead of the original side view 52. (see FIG. 12). Note that when a leftward drag operation is performed, the side view 52 rotated in the opposite direction around AX1 is displayed.

On the other hand, when the instruction movement amount ΔU is smaller than the instruction movement amount ΔV (see FIG. 13), it is determined that the instruction gesture is not a view correction instruction. In other words, “do not rotate (change) the viewpoint” is specified as the instruction content of the instruction gesture for the operation target view 52 . In this case, correction processing of the side view 52 is not performed (see FIG. 4).

<Change of Viewpoint of Top View 53>
Further, as shown in FIGS. 14 to 16, when the user performs a drag operation (instruction gesture) from the state where the mouse cursor 65 exists on the top view 53, the following processing is executed. Note that a rightward drag operation is performed in FIG. 14 , an upward drag operation is performed in FIG. 15 , and an upper left drag operation is performed in FIG. 16 .

In this case, first, it is determined based on the second information F2 of the pixel that the pixel at the operation start position of the drag operation is associated with the top view 53, and the view specifying unit 24 determines that the operation target view is the top view. It is determined that it is the view 53 (step S14).

Next, the instruction content of the instruction gesture is identified regardless of the magnitude relationship between the instruction movement amount ΔU and the instruction movement amount ΔV (see FIG. 6). Specifically, it is specified that an instruction to rotate the viewpoint of the top view 53 around the vertical axis AX1 in the three-dimensional space has been given (step S15). Then, a view image obtained by rotating the viewpoint of the top view 53 around the vertical axis AX1 in the three-dimensional space is generated as a new top view based on the instruction content of the instruction gesture (step S16). Then, the new top view 53 is displayed on the screen 200 in place of the original top view 53 (see FIG. 17). In the top view 53, the vertical axis AX1 in the three-dimensional space extends in a direction perpendicular to the screen 200 (paper surface). Note that when a reverse rotation operation is performed with respect to the central axis of the top view 53, the top view 53 rotated in the reverse direction around AX1 is displayed.

<Exception handling in drag operation>
When a drag operation is performed on one of a plurality of views and the drag operation is performed with a starting point other than a predetermined mark (e.g., isodepth line), the above-described processing is performed.

On the other hand, even when a drag operation is performed on one of a plurality of views, if the drag operation is performed with a predetermined mark (e.g. depth contour) as the starting point, the following exceptional processing is performed. is done.

For example, as shown in FIG. 19, a vertical (eg, downward) drag from a position on a contour line (curve indicating the same depth position in the depth direction in water) 63 (63a) in perspective view 51. When an operation is performed, processing is performed to move the contour lines 63 in the fluoroscopic view 51 . FIG. 20 shows how the contour lines 63 (63a) are moved downward on the screen 200 after the change.

<Moving operation and resizing operation>
The screen 200 is provided with a mode transition button 69 (see FIG. 4, etc.). Each time the mode transition button 69 is pressed, the operation mode is switched between the "rotation mode" and the "arrangement change mode". The "rotation mode" is a mode for realizing the above-described operations (rotating operations of the respective views 51, 52, 53) (see FIGS. 4 to 20). In this mode, it is possible to change the arrangement of the views 51, 52, 53 (see FIG. 21, etc.).

When the mode transition button 69 in the screen 200 is pressed and the operation mode transitions to the "arrangement change mode", the following operations are performed.

Specifically, the position of each operation target view within the screen 200 is changed according to the drag operation on the operation target view. For example, as shown in FIG. 21, the position of the top view 53 within the screen 200 is changed to the lower left side in response to the downward left drag operation on the top view 53 . In FIG. 21 , the top view 53 has been moved to a position near the lower right of the fluoroscopic view 51 and near the upper right of the side view 52 .

Furthermore, when a mouse wheel operation is performed while the mouse cursor is in the operation target view, the size of each operation target view itself within the screen 200 is changed according to the wheel operation. For example, as shown in FIG. 22, when the mouse wheel is rotated with the mouse cursor 65 on the fluoroscopic view 51, it is determined that an instruction to enlarge the fluoroscopic view 51 has been given. View 51 is enlarged. In other words, the proportion of the perspective view 51 within the screen 200 is increased. Conversely, when the mouse wheel operation is performed in the opposite direction (rotation operation to the near side), the fluoroscopic view 51 is reduced. The same applies to other views 52 and 53 as well.

Each view is scaled (enlarged and/or reduced) around the position of the mouse cursor 65 . However, it is not limited to this, and may be scaled (magnified) around a predetermined reference position (for example, the position of the ship S).

The user can freely change the size and position of each view by combining the above-described movement operation and enlargement/reduction operation (see FIG. 23). In other words, the user can arrange each view in any position and any size within the display screen 200 . Note that FIG. 23 shows a state in which the size and position of each view are changed based on a move operation and an enlargement/reduction operation on the operation target views 51 and 52 and a move operation on the top view 53 .

In addition, the user can freely change the size and position of each view by combining the operation in the "arrangement change mode" and the operation in the "rotation mode", and can freely change the viewpoint of each view image. It is possible to show it to the user (such as rotating the perspective view 51 around the axis AX1).

The user can also specify the display position and display size of each view in the state immediately after restarting the device (initial state) by a similar operation.

<1-7. Effects of First Embodiment>
As described above, each pixel included in the plurality of views 50 (51, 52, 53) is associated with the first information F1 displayed on the screen 200 and the destination of each pixel among the plurality of views 50. It is associated with a plurality of information including second information F2 indicating the view. Therefore, based on the second information F2, it is possible to easily obtain information about the view to which each pixel belongs. As a result, it is possible to more appropriately determine the gesture target area from among the plurality of areas respectively corresponding to the plurality of views. More specifically, the operation target view, which is the operation target view, is specified based on the second information F2 associated with the target pixel of the operation by the user, thereby obtaining a plurality of views corresponding to the plurality of views. It is possible to better determine the target area of the gesture from within the area.

In particular, even when two adjacent views (apparently) overlap each other (see FIG. 23), the view to which each pixel belongs can be determined appropriately by using the second information F2 of each pixel in the overlap region. is determined to Therefore, even when the target pixel of the user's operation is a pixel within the overlapping region, the operation target view (gesture's view) is based on the second information F2 associated with the target pixel. region of interest) can be determined better.

Also, as described above, each view can have various shapes other than a rectangle (a shape in which two halves of an ellipse are combined vertically, a lower semi-circular shape, a circular shape, etc.). That is, it is possible to improve the flexibility of the shape. Furthermore, as described above, the second information F2 (view identification information) is associated with each pixel and stored, so that the view to be operated can be easily identified while improving the degree of freedom in shape. Is possible.

In this way, it is possible to more appropriately determine the operation target view (gesture target area) while improving the degree of freedom in the shape of each view and the degree of freedom in arrangement of each view.

Further, in the above embodiment, the view to be operated is automatically specified according to the position of the mouse cursor, so the user presses a predetermined button (button for switching view to be operated) to switch the view to be operated. No action required. Therefore, the user can easily specify the view to be operated.

In addition, in the above-described embodiment, the view to be operated is specified according to the position of the mouse cursor, and the action in response to the mouse operation is automatically changed according to the view to be operated. For example, the content of the viewpoint change operation (object rotation operation) is changed depending on whether the same downward drag operation is performed on the perspective view 51 or the top view 53 . Specifically, when the downward drag operation is performed on the perspective view 51, the viewpoint is rotated around the axis AX2, and when the downward drag operation is performed on the top view 53, the axis AX1 is rotated. A viewpoint rotation operation is performed around. In this way, appropriate actions can be automatically performed depending on the view.

<1-8. Modified example according to the first embodiment>
In the first embodiment described above, the view (view image) correction processing is executed only for the operation target view, and the view correction processing is not changed for views other than the operation target view. However, the invention is not so limited.

Specifically, when a view modification instruction is given to a manipulation target view, not only the manipulation target view but also at least one of the plurality of views other than the manipulation target view may be modified. . In other words, view correction processing may be performed on views other than the operation target view in conjunction with an operation instruction for the operation target view.

For example, when the view to be operated is the perspective view 51, and a view correction instruction is given to the perspective view 51 to rotate the viewpoint of the perspective view 51 around the axis AX1 and the perspective view 51 is updated (FIG. 7 ), the screen 200 may be modified as shown in FIG. Specifically, the viewpoint of the perspective view 51 is rotated around the axis AX1 (see FIG. 7 etc.) to update the perspective view 51, and the viewpoint of the side view 52 is rotated around the axis AX1 (see FIG. 11 etc.). may also update the side view 52 (see FIG. 18). Furthermore, the top view 53 may also be updated by rotating the top view 53 around the axis AX1 (see FIG. 14, etc.) (see FIG. 18).

<2. Second Embodiment>
The second embodiment is a modification of the first embodiment. Below, it demonstrates centering around difference with 1st Embodiment.

In the second embodiment, a touch panel type display device (hereinafter simply referred to as a touch panel) is provided as the display unit 31, and a gesture (also referred to as a touch gesture) using the touch panel is accepted. In the second embodiment, a touch panel is provided as the user interface section. In other words, the touch panel functions as a display unit and also as an operation input unit.

In the second embodiment, a one-finger drag operation in the touch gesture gives an instruction similar to the instruction by the drag operation in the mouse gesture of the first embodiment.

In addition, in the second embodiment, a two-finger operation (an operation performed with two fingers at the same time (also referred to as a multi-touch operation)) in touch gestures is also accepted. The two-finger operation includes a pinch-in operation, a pinch-out operation, a two-finger drag operation, and a two-finger rotation operation.

A pinch-in operation is an operation to narrow the distance between two fingers on the screen, and a pinch-out operation is an operation to widen the distance between two fingers on the screen. A two-finger drag operation (also referred to as a slide operation or a swipe operation) is an operation in which two fingers are simultaneously moved in a certain direction (the same direction) on the screen. In addition, the two-finger rotation operation can be performed by moving the two fingers on the screen in accordance with the movement of the two fingers (moving the two fingers in opposite directions around a certain position on the screen, etc.). This is an operation to change the direction in which positions are connected to each other.

In the second embodiment, among a plurality of gestures, a multi-touch gesture (specifically, a pinch-in operation, a pinch-out operation, and a two-finger drag operation) is used to achieve the "rotation mode" and the "arrangement change". It is possible to change the position of the view to be operated and change the scaling ratio without switching the operation mode.

Specifically, when a two-finger drag operation is performed on the operation target view, it is determined that an instruction to move the operation target view (an instruction to change the display position within the screen) has been given. Then, the operation target view (for example, the top view 53) is moved to the position desired by the user according to the movement instruction (see FIG. 21). Note that the two-finger drag operation is identified as a move instruction operation for moving the operation target view within the screen regardless of the type of the operation target view.

Further, when a two-finger pinch operation (pinch-in operation/pinch-out operation) is performed on the operation target view, an instruction to change the magnification of the operation target view (an instruction to change the display size (display area) within the screen) is performed. is determined to have been given. Then, the size of the operation target view is changed according to the scaling instruction. Specifically, the pinch-out operation is determined to be an instruction to enlarge the view to be operated, and the view to be operated (for example, perspective view 51) is enlarged according to the pinch-out operation (see FIG. 22). On the other hand, the pinch-in operation is determined to be an instruction to reduce the view to be operated, and the view to be operated is reduced according to the pinch-in operation. It should be noted that the two-finger pinch operation is specified as a zoom instruction operation for zooming the operation target view within the screen regardless of the type of the operation target view.

In addition, a two-finger rotation operation is specified as an operation of giving a “rotation instruction about the vertical axis AX1” regardless of the type of the operation target view. That is, even if the operation target view of the two-finger rotation operation is any one of the perspective view 51, the side view 52, and the top view 53, the two-finger rotation operation is a "rotation about the vertical axis AX1". It is determined that it is an "instruction" operation.

Further, when a one-finger drag operation is applied to each operation target view, it is determined that an instruction similar to the mouse drag operation in the "rotation mode" is applied.

Specifically, when a one-finger drag operation is performed as a touch operation on the perspective view 51, and the instruction movement amount ΔU in the drag operation is larger than the instruction movement amount ΔV, the "rotation instruction about the vertical axis AX1" is specified as the instruction content of the instruction gesture (see FIGS. 7 and 8). In a similar case, when the instruction movement amount ΔU in the drag operation is smaller than the instruction movement amount ΔV, "rotation instruction about the horizontal axis AX2" is specified as the instruction content of the instruction gesture (FIGS. 9 and 10). reference).

Further, when a one-finger drag operation is performed as a touch operation on the side view 52, and the instruction movement amount ΔU in the drag operation is larger than the instruction movement amount ΔV, "rotation instruction about the vertical axis AX1" is performed. is specified as the instruction content of the instruction gesture (see FIGS. 11 and 12). In a similar case, when the instruction movement amount ΔU in the drag operation is smaller than the instruction movement amount ΔV (see FIG. 13), "not to change the viewpoint" is specified as the instruction content of the instruction gesture.

Further, when a one-finger drag operation is performed as a touch operation on the top view 53, regardless of the magnitude relationship between the instruction movement amount ΔU and the instruction movement amount ΔV in the drag operation, the "rotation about the vertical axis AX1 "instruction" is specified as the instruction content of the pointing gesture (see FIGS. 14 to 17).

Effects similar to those of the first embodiment can be obtained by such a second embodiment.

Further, in the second embodiment, the operation target view is automatically specified according to the touch start position of the touch gesture. More specifically, the operation target view is automatically identified based on the second information F2 associated with the pixel of the touch start position. Therefore, the user does not need to press the predetermined button each time the view to be operated is switched. Therefore, the user can easily specify the view to be operated.

In addition, in the second embodiment, different instruction contents are assigned to two-finger operation (multi-touch operation) and one-finger operation (single-touch operation). Then, by properly using a plurality of types of touch operations, a large number of instruction contents are imparted while being distinguished from each other. Therefore, it is not necessary to perform a mode change operation by pressing the mode transition button 69 or the like. Specifically, the viewpoint change operation (rotation operation) of each view, and the movement operation and/or magnification change operation of each view can be realized only by touch operations.

<3. Modifications, etc.>
Although the embodiments of the present invention have been described above, the present invention is not limited to the contents described above.

For example, in the above-described first embodiment and the like, there is illustrated a mode in which the movement and/or magnification instruction of the operation target view is accompanied by an operation other than the mouse operation (specifically, the operation of pressing the mode transition button 69). However, it is not limited to this, and a combination of a mouse button operation, a mouse movement operation, and a mouse wheel operation may be used to distinguish the operation target view movement and/or scaling instruction. In detail, "moving the view" is instructed by moving the mouse while pressing both the left and right mouse buttons, and "zooming (magnifying) the view" is performed by rotating the mouse wheel while pressing the left mouse button. and reduction)" may be indicated. This also allows various kinds of instructions to be given without pressing the mode transition button 69 .

Further, in each of the above-described embodiments and the like, an underwater detection device having a transducer that functions as a transmitter and a receiver is mainly exemplified, but the underwater detection device is not limited to this. , the transmitting unit and the receiving unit may be separately provided.

In each of the above-described embodiments and the like, a data set is acquired using a scanning sonar that simultaneously forms transmission beams in all directions underwater centering on the ship, but the present invention is not limited to this. For example, the dataset may be acquired using an underwater sounding device with a searchlight sonar (PPI sonar) that rotates the transmit and receive beams.

In addition, in each of the above-described embodiments and the like, a mode in which a data set (fish school data, etc.) detected from the underwater environment regarding an underwater target (fish school, etc.) is acquired is illustrated, but the present invention is not limited to this. For example, a sensed data set (such as meteorological data) from the airborne environment may be obtained for an airborne target (such as water vapor). Alternatively, a data set (such as medical data) detected from the internal environment for a target (such as an organ) within the human body may be obtained.

1 Underwater detection device (data processing device)
10 scanning sonar 31 display unit 33 operation input unit 51 fluoroscopic view 52 side view 53 top view 63 contour lines 65 mouse cursor 68 pixels 69 mode transition button 200 display screen AX1 vertical axis AX2 horizontal axis F1 first information F2 second information

Claims (15)

A data processing device,
a data acquisition unit for acquiring a data set detected by the detection device for the target;
a rendering unit that executes rendering processing based on the data set and generates a plurality of views arranged within a screen;
with
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
a user interface unit that receives a user's operation on the plurality of views;
View identification for acquiring the second information associated with the target pixel of the operation by the user and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel. Department and
a gesture detection unit that detects a gesture related to the operation as an instruction gesture;
a view correction unit that corrects the operation target view based on the instruction gesture when the instruction gesture is a view correction instruction;
further comprising
wherein the plurality of views includes perspective views;
When the operation target view is the perspective view and the amount of movement of the instruction gesture in the horizontal direction within the screen is larger than the amount of movement of the instruction gesture in the vertical direction within the screen , a rotated view of the perspective view about the first axis in the three-dimensional space for the data set, which in the perspective view corresponds to the vertical direction in the screen. A data processing device for generating an image as a new perspective view. A data processing device,
a data acquisition unit for acquiring a data set detected by the detection device for the target;
a rendering unit that executes rendering processing based on the data set and generates a plurality of views arranged within a screen;
with
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
a user interface unit that receives a user's operation on the plurality of views;
View identification for acquiring the second information associated with the target pixel of the operation by the user and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel. Department and
a gesture detection unit that detects a gesture related to the operation as an instruction gesture;
a view correction unit that corrects the operation target view based on the instruction gesture when the instruction gesture is a view correction instruction;
further comprising
wherein the plurality of views includes perspective views;
When the operation target view is the perspective view and the movement amount of the instruction gesture in the horizontal direction within the screen is smaller than the movement amount of the instruction gesture in the vertical direction within the screen , a rotated view of the perspective view about the second axis in the three-dimensional space for the data set, which in the perspective view corresponds to the horizontal direction in the screen. A data processing device for generating an image as a new perspective view. A data processing device,
a data acquisition unit for acquiring a data set detected by the detection device for the target;
a rendering unit that executes rendering processing based on the data set and generates a plurality of views arranged within a screen;
with
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
a user interface unit that receives a user's operation on the plurality of views;
View identification for acquiring the second information associated with the target pixel of the operation by the user and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel. Department and
a gesture detection unit that detects a gesture related to the operation as an instruction gesture;
a view correction unit that corrects the operation target view based on the instruction gesture when the instruction gesture is a view correction instruction;
further comprising
the plurality of views includes a side view;
When the operation target view is the side view and the movement amount of the instruction gesture in the horizontal direction within the screen is larger than the movement amount of the instruction gesture in the vertical direction within the screen , a view obtained by rotating the viewpoint of the side view about the first axis that is the first axis in the three-dimensional space for the data set and that corresponds to the vertical direction in the screen in the side view. A data processing device characterized by generating an image as a new side view. A data processing device,
a data acquisition unit for acquiring a data set detected by the detection device for the target;
a rendering unit that executes rendering processing based on the data set and generates a plurality of views arranged within a screen;
with
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
a user interface unit that receives a user's operation on the plurality of views;
View identification for acquiring the second information associated with the target pixel of the operation by the user and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel. Department and
a gesture detection unit that detects a gesture related to the operation as an instruction gesture;
a view correction unit that corrects the operation target view based on the instruction gesture when the instruction gesture is a view correction instruction;
further comprising
the plurality of views includes a side view;
When the operation target view is the side view and the movement amount of the instruction gesture in the horizontal direction within the screen is smaller than the movement amount of the instruction gesture in the vertical direction within the screen , the data processing device, wherein the viewpoint of the side view is not rotated. A data processing device,
a data acquisition unit for acquiring a data set detected by the detection device for the target;
a rendering unit that executes rendering processing based on the data set and generates a plurality of views arranged within a screen;
with
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
a user interface unit that receives a user's operation on the plurality of views;
View identification for acquiring the second information associated with the target pixel of the operation by the user and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel. Department and
a gesture detection unit that detects a gesture related to the operation as an instruction gesture;
a view correction unit that corrects the operation target view based on the instruction gesture when the instruction gesture is a view correction instruction;
further comprising
the plurality of views includes a top view;
When the operation target view is the top view and an instruction to move in the horizontal direction or the vertical direction within the screen is given based on the instruction gesture, the view modification unit performs a movement instruction in a three-dimensional space related to the data set. generating, as a new top view, a view image in which the viewpoint of the top view is rotated around the first axis which is the first axis and corresponds to the direction perpendicular to the screen in the top view. A data processing device characterized by: A data processing method for obtaining a data set detected by a detection device about a target and performing a rendering process based on the data set to generate a plurality of views in a screen, comprising:
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
Receiving a user's operation on the plurality of views,
obtaining the second information associated with the target pixel of the operation by the user, and identifying the operation target view, which is the target view of the operation, based on the second information of the target pixel;
detecting a gesture related to the operation as a pointing gesture;
if the instruction gesture is a view modification instruction, modifying the operation target view based on the instruction gesture;
wherein the plurality of views includes perspective views;
When the operation target view is the perspective view and the amount of movement of the instruction gesture in the horizontal direction within the screen is larger than the amount of movement of the instruction gesture in the vertical direction within the screen, 3 regarding the data set A view image obtained by rotating the viewpoint of the perspective view around the first axis which is the first axis in the dimensional space and which corresponds to the vertical direction in the screen in the perspective view is used as a new perspective view. A data processing method characterized by generating as A data processing method for obtaining a data set detected by a detection device about a target and performing a rendering process based on the data set to generate a plurality of views in a screen, comprising:
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
Receiving a user's operation on the plurality of views,
obtaining the second information associated with the target pixel of the operation by the user, and identifying the operation target view, which is the target view of the operation, based on the second information of the target pixel;
detecting a gesture related to the operation as a pointing gesture;
if the instruction gesture is a view modification instruction, modifying the operation target view based on the instruction gesture;
wherein the plurality of views includes perspective views;
When the operation target view is the perspective view and the amount of movement of the pointing gesture in the horizontal direction within the screen is smaller than the amount of movement of the pointing gesture in the vertical direction within the screen, 3 regarding the data set A view image obtained by rotating the viewpoint of the perspective view around the second axis which is a second axis in the dimensional space and which corresponds to the horizontal direction in the screen in the perspective view is added to a new perspective view. A data processing method characterized by generating as A data processing method for obtaining a data set detected by a detection device about a target and performing a rendering process based on the data set to generate a plurality of views in a screen, comprising:
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
Receiving a user's operation on the plurality of views,
obtaining the second information associated with the target pixel of the operation by the user, and identifying the operation target view, which is the target view of the operation, based on the second information of the target pixel;
detecting a gesture related to the operation as a pointing gesture;
if the instruction gesture is a view modification instruction, modifying the operation target view based on the instruction gesture;
the plurality of views includes a side view;
When the operation target view is the side view and the amount of movement of the pointing gesture in the horizontal direction within the screen is larger than the amount of movement of the pointing gesture in the vertical direction within the screen, 3 A view image obtained by rotating the viewpoint of the side view around the first axis which is the first axis in the dimensional space and which corresponds to the vertical direction in the screen in the side view is provided as a new side view. A data processing method characterized by generating as A data processing method for obtaining a data set detected by a detection device about a target and performing a rendering process based on the data set to generate a plurality of views in a screen, comprising:
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
Receiving a user's operation on the plurality of views,
obtaining the second information associated with the target pixel of the operation by the user, and identifying the operation target view, which is the target view of the operation, based on the second information of the target pixel;
detecting a gesture related to the operation as a pointing gesture;
if the instruction gesture is a view modification instruction, modifying the operation target view based on the instruction gesture;
the plurality of views includes a side view;
When the operation target view is the side view and the amount of movement of the pointing gesture in the horizontal direction within the screen is smaller than the amount of movement of the pointing gesture in the vertical direction within the screen, the viewpoint of the side view is not rotated. A data processing method for obtaining a data set detected by a detection device about a target and performing a rendering process based on the data set to generate a plurality of views in a screen, comprising:
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
Receiving a user's operation on the plurality of views,
obtaining the second information associated with the target pixel of the operation by the user, and identifying the operation target view, which is the target view of the operation, based on the second information of the target pixel;
detecting a gesture related to the operation as a pointing gesture;
if the instruction gesture is a view modification instruction, modifying the operation target view based on the instruction gesture;
the plurality of views includes a top view;
When the operation target view is the top view and an instruction to move in the horizontal direction or the vertical direction within the screen is given based on the pointing gesture, the first axis in the three-dimensional space related to the data set A data processing method comprising: generating, as a new top view, a view image obtained by rotating the viewpoint of the top view about the first axis associated with the direction perpendicular to the screen in the top view. . to the computer,
a) acquiring a data set detected by a detection device for a target;
b) performing a rendering process based on said dataset to generate multiple views within a screen;
A program for executing
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
to the computer;
c) accepting user manipulation of the plurality of views;
d) obtaining the second information associated with the target pixel of the operation by the user, and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel; a step;
e) detecting a gesture relating to said operation as a pointing gesture;
f) if the pointing gesture is a view modifying instruction, modifying the operated view based on the pointing gesture;
further execute
wherein the plurality of views includes perspective views;
In the step of correcting the operation target view, the operation target view is the perspective view, and the amount of movement of the pointing gesture in the horizontal direction within the screen is the amount of movement of the pointing gesture in the vertical direction within the screen. If greater than A program characterized by generating a rotated view image as a new perspective view. to the computer,
a) acquiring a data set detected by a detection device for a target;
b) performing a rendering process based on said dataset to generate multiple views within a screen;
A program for executing
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
to the computer;
c) accepting user manipulation of the plurality of views;
d) obtaining the second information associated with the target pixel of the operation by the user, and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel; a step;
e) detecting a gesture relating to said operation as a pointing gesture;
f) if the pointing gesture is a view modifying instruction, modifying the operated view based on the pointing gesture;
further execute
wherein the plurality of views includes perspective views;
In the step of correcting the operation target view, the operation target view is the perspective view, and the amount of movement of the pointing gesture in the horizontal direction within the screen is the amount of movement of the pointing gesture in the vertical direction within the screen. If less than A program characterized by generating a rotated view image as a new perspective view. to the computer,
a) acquiring a data set detected by a detection device for a target;
b) performing a rendering process based on said dataset to generate multiple views within a screen;
A program for executing
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
to the computer;
c) accepting user manipulation of the plurality of views;
d) obtaining the second information associated with the target pixel of the operation by the user, and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel; a step;
e) detecting a gesture relating to said operation as a pointing gesture;
f) if the pointing gesture is a view modifying instruction, modifying the operated view based on the pointing gesture;
further execute
the plurality of views includes a side view;
In the step of correcting the operation target view, the operation target view is the side view, and the amount of movement of the pointing gesture in the horizontal direction within the screen is the amount of movement of the pointing gesture in the vertical direction within the screen. If greater than A program characterized by generating a rotated view image as a new side view. to the computer,
a) acquiring a data set detected by a detection device for a target;
b) performing a rendering process based on said dataset to generate multiple views within a screen;
A program for executing
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
to the computer;
c) accepting user manipulation of the plurality of views;
d) obtaining the second information associated with the target pixel of the operation by the user, and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel; a step;
e) detecting a gesture relating to said operation as a pointing gesture;
f) if the pointing gesture is a view modifying instruction, modifying the operated view based on the pointing gesture;
further execute
the plurality of views includes a side view;
In the step of correcting the operation target view, the operation target view is the side view, and the amount of movement of the pointing gesture in the horizontal direction within the screen is the amount of movement of the pointing gesture in the vertical direction within the screen. , wherein the side view viewpoint is not rotated if less than . to the computer,
a) acquiring a data set detected by a detection device for a target;
b) performing a rendering process based on said dataset to generate multiple views within a screen;
A program for executing
each of the plurality of views includes a plurality of pixels;
Each pixel included in the plurality of views is a plurality of pieces of information including first information displayed on the screen and second information indicating a view to which each pixel belongs among the plurality of views. is mapped to
to the computer;
c) accepting user manipulation of the plurality of views;
d) obtaining the second information associated with the target pixel of the operation by the user, and specifying the operation target view, which is the target view of the operation, based on the second information of the target pixel; a step;
e) detecting a gesture relating to said operation as a pointing gesture;
f) if the pointing gesture is a view modifying instruction, modifying the operated view based on the pointing gesture;
further execute
the plurality of views includes a top view;
In the step of correcting the operation target view, when the operation target view is the top view and an instruction to move in the horizontal direction or the vertical direction within the screen is given based on the pointing gesture, 3 regarding the data set: A view image obtained by rotating the viewpoint of the top view around the first axis that is the first axis in the dimensional space and corresponds to the direction perpendicular to the screen in the top view is used as a new top view. A program characterized by generating

Images