JP4373517B2 - Information processing apparatus and information processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an information processing apparatus and an information processing method for processing and outputting an input image in virtual reality, and removes a burden on a body such as a headphone and a head mounted display, and enhances an image effect and a sound effect. The present invention relates to an information processing apparatus and an information processing method.
[0002]
In recent years, in an amusement computer whose demand is increasing, a virtual reality (hereinafter referred to as a “VR device”) in which the world projected by video is captured by the illusion of being experienced by itself. Devices that can be experienced are gaining popularity at theme parks. The virtual reality device creates a three-dimensional virtual world on the computer screen with stereoscopic CG (computer graphics), and the user becomes a central person by manipulating the virtual world. You can have an experience with.
[0003]
However, a virtual reality device (hereinafter referred to as “VR device”) is a business device installed in a theme park or the like as an expensive simulation device or an amusement device.
In the future, there is a demand for a device that can spread the virtual reality to the general public.
[0004]
[Prior art]
In the conventional VR technology, the following control method is common.
FIG. 1 is an external view of a conventional example.
As shown in FIG. 1, in the conventional VR technology, video and audio are controlled by a control computer 1. The operator 5 puts a VR dedicated device on his head. The sound is transmitted from the control computer 1 to the operator 5 through the headphones 4. In addition, the same image as the image displayed on the liquid crystal display is displayed on the HMD 2 (head mounted display) by the control computer 1. The control computer 1 controls an image displayed on the HMD 2 in accordance with the movement of the line of sight of the operator 5 sensed by the sensor 3.
[0005]
As described above, the conventional VR technology creates and expresses a virtual world by putting a special device on the head.
[0006]
[Problems to be solved by the invention]
However, the conventional VR technology has the following problems.
First, in FIG. 1, the HMD 2 worn by the operator 5 is heavy and places a burden on the head and neck, so there is a disadvantage that it cannot be used for a long time.
[0007]
In addition, the HMD 2 is a special device including an expensive sensor for sensing eye movement, so that the initial purchase cost is high and the maintenance is expensive.
Furthermore, the HMD 2 has a feature that the outside world is blocked and the 360 ° omnidirectional image is displayed in accordance with the direction of the line of sight of the operator 5, but the 360 ° omnidirectional direction is clearly displayed, which gives the operator 5 a visual burden. There may be.
[0008]
Next, the headphone 4 worn by the operator 5 applies a load to the ear, and further imposes a burden on the body of the operator 5 in addition to the HMD 2.
An object of the present invention has been made in view of the above problems, and is an information processing having an image and sound technology that removes a burden on the body such as a headphone and a head-mounted display and enhances the image effect and the sound effect. An object is to provide an apparatus and an information processing method.
[0009]
[Means for Solving the Problems]
The present invention has been made in view of the above problems, and is characterized by taking the following means. According to the first aspect of the present invention, in the information processing apparatus that processes and outputs an input image, the input image is divided into a right screen, a center screen, and a left screen, and the divided right image is displayed on the right screen. A right image display means for displaying, a central image display means for displaying the divided central image on the central screen, and a left image display means for displaying the divided left image on the left screen ; Upper right audio reproduction means for dividing audio data corresponding to the divided right screen, center screen, and left screen, and reproducing audio in the upper right direction, and lower right for reproducing audio in the lower right screen Audio reproduction means, central upper audio reproduction means for reproducing audio in the upper direction of the central screen, lower central audio reproduction means for reproducing audio in the lower direction of the central screen, and upper left audio reproduction means for reproducing audio in the upper direction of the left screen And the sound of the lower left screen A voice dividing means Ru is reproduced output to the lower left audio reproducing means for reproducing the right screen, a pointing device for designating a desired position of the center screen, and displayed on the left screen and the said input image, the pointing predetermined radius range using all scan lines, and image exaggerates the image exaggeration processing means in the range of the predetermined radial performs image data processing so strong brightness around the position coordinates specified by the device Sound position calculating means for calculating a sound position from the position coordinates, and sound enhancement processing means for exaggerating the sound by adding information for increasing the sound pressure to the sound position in the sound data , display splitting means, the image data processing by dividing the input image is performed, the right image display means to display the left image display means, and the central image display means It, the operator does not need to be equipped with special equipment to the body, and, it is possible to see the image you want often seen clearly. In addition, since the position of the image information required by the operator and the sound information are associated with each other, it is possible to express a virtual space that is closer to human visual functions and auditory functions , and to obtain information such as voices that are often heard. Ru can be obtained. Furthermore, it is possible to easily specify the position of the image by the operator using the pointing device.
[0010]
In the invention of claim 2, comprising a detection means for detecting the position coordinates of the line of sight of the user, the image exaggeration processing means, line-of-sight detection in the pointing device specified by the position coordinates or the detection means Since the image data processing is performed based on the position coordinates, it is possible to easily specify the position of the image by the operator.
[0011]
According to a third aspect of the present invention, the image exaggeration processing means performs the image data processing so as to thin out the scanning lines to be displayed for each predetermined line outside the range of the predetermined radius, so that image information required by the operator is obtained. Can be exaggerated and image information required by the operator can be accurately conveyed.
[0014]
According to a fourth aspect of the present invention, the sound exaggeration processing means can exaggerate and convey the sound information required by the operator by adding information that lowers the sound pressure except at the sound position.
[0016]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
FIG. 2 is a functional configuration diagram of an embodiment of the present invention.
As shown in FIG. 2, the VR apparatus includes a DVD 101 that reads recorded data, a control unit 100 that controls a display image and sound, a display unit 120 that displays an image, and an audio unit 140 that reproduces sound.
[0017]
The control unit 100 is divided into an image data decompression unit 102 that decompresses the compressed image data of the DVD 101, an exaggeration processing unit 103 that locally exaggerates the image, and an image data division unit 104 that divides the image into right, center, and left images. A right image data processing unit 105 for processing right image data, a central image data processing unit 106 for processing divided central image data, a left image data processing unit 107 for processing divided left image data, and a DVD 101 Audio data decompressing unit 108 for decompressing the compressed audio data, audio data dividing unit 109 for dividing the audio data into right, center and left images respectively, and right image audio data processing for processing the divided audio data for the right image 110, a central image audio data processing unit 111 for processing the audio data for the divided central image, and an audio data for the divided left image. A left image and sound data processing unit 112, the coordinate detection processing unit 113 for detecting an exaggerate image position for processing data, and a CPU114 for controlling the image processing and audio processing according to the image position.
[0018]
The display unit 120 displays a right image display unit 121 that displays a right image divided on the right screen, a central image display unit 122 that displays a central image divided on the central screen, and a left image divided on the left screen. The left image display unit 123 and a pointing device 113a for designating the coordinate position of the screen are configured.
The right image display unit 121, the center image display unit 122, and the left image display unit 123 display an image on a liquid crystal display, for example.
[0019]
The audio unit 140 includes an upper right audio reproduction unit 141 that reproduces audio in the upper right direction, a lower right audio reproduction unit 142 that reproduces audio in the lower right direction, and an upper audio reproduction unit that reproduces audio in the upper central screen. 143, lower center audio reproduction unit 144 that reproduces audio in the lower direction of the central screen, upper left audio reproduction unit 145 that reproduces audio in the upper direction of the left screen, lower left audio reproduction unit 146 that reproduces audio in the lower direction of the left screen, operator's It comprises a line-of-sight monitoring camera 513b that monitors the line of sight and detects the position of the operator's line of sight on the screen.
[0020]
The sound is reproduced by a Dolby sound method or the like.
In the control unit 100, the CPU 114 decompresses the image data recorded on the DVD 101 by the image data decompression unit 102, and decompresses the image data by the image data decompression unit 102 corresponding to the coordinate position on the screen detected by the coordinate detection processing unit 113. The image data is extracted, and the image data extracted by the exaggeration processing unit 103 is controlled to be exaggerated. Thereafter, the image data dividing unit 104 divides all image data including the image data subjected to the exaggeration processing into right, center, and left screens, which are processed and displayed by the respective data processing units 105, 106, and 107. Displayed on the unit 120.
[0021]
The exaggeration processing unit 103 performs processing so that the range of the predetermined radius R from the coordinate position detected by the coordinate detection processing unit 113 is a clear image and the range other than the range of the predetermined radius R is blurred.
For example, for image data outside the range of a predetermined radius R, the image can be blurred by performing image data processing such as thinning out the scanning lines to be displayed every other line. As a result, the image in the range of the predetermined radius R becomes a clear and clear image.
[0022]
Further, by performing the exaggeration process before dividing the image data, it is not necessary to perform the process for each screen.
The control unit 100 uses the CPU 114 to decompress the audio data recorded on the DVD 101 by the audio data decompression unit 108 and divide the audio data decompressed by the audio data division unit 109 for each screen. The data processing units 110, 112, and 113 are controlled to perform processing. In addition, the CPU 114 instructs the sound data processing units 110, 111, and 112 to select a sound pressure corresponding to the coordinate position detected by the coordinate detection processing unit 113. Thereafter, the sound unit 140 reproduces sound that has been subjected to sound pressure processing for each screen.
[0023]
The coordinate detection processing unit 113 detects the coordinate position by the pointing device 113a and / or the line-of-sight monitoring camera 113b.
FIG. 3 is a diagram showing the appearance of an embodiment of the present invention.
The same reference numerals are given to the same components as those already described, and the description thereof is omitted.
[0024]
As shown in FIG. 3, the screen of the VR apparatus main body 160 is divided into an area 200 where an image is clearly displayed and an area where the image is not clearly displayed.
The area 200 is an image within a predetermined radius R from the coordinate position designated by the pointing device 113a or the line-of-sight monitoring camera 113b, and is clearly displayed.
[0025]
On the other hand, areas other than the area 200 are subjected to image data processing such as thinning out scanning lines to be displayed every other line, and a blurred image is displayed.
In this way, it is possible to reduce physical fatigue caused by the operator's vision by adjusting the image and clarifying only necessary information.
FIG. 4 is a top view of one embodiment of the present invention.
[0026]
The same reference numerals are given to the same components as those already described, and the description thereof is omitted.
As shown in FIG. 4, the VR apparatus main body 160 is installed with the screens in contact with each other, that is, the angle between the right image display unit 121 and the central image display unit 122, and the left image display unit 123 and the central image display unit 122 being θ. The
[0027]
The angle θ is recommended to be 120 degrees, but can be adjusted according to the use of the operator.
FIG. 5 is an external view showing the operation of one embodiment of the present invention.
As shown in FIG. 5, the VR device main body 160 is large enough to be placed on a desk 21 such as an office work desk or a study desk, and an operator sits on a chair 22 and uses the VR device of the present invention. Can do.
[0028]
Next, a processing flow for realizing the VR device of the present invention will be described.
FIG. 6 is a flowchart of image exaggeration processing according to an embodiment of the present invention.
As shown in FIG. 6, in the image exaggeration process, step S11 for detecting the operator's line-of-sight position, step S12 for calculating an image within the radius R around the coordinates (x, y), and the calculation result to the image exaggeration processing unit. And output step S13.
[0029]
In step S <b> 11 for detecting the operator's line-of-sight position, the CPU 114 controls to fetch information indicating the operator's line-of-sight position from the coordinate detection processing unit 113.
In step S12 for calculating the image within the radius R around the coordinates (x, y), the line-of-sight position information captured in step S11 is converted into coordinates, and the range of the predetermined radius R around the coordinates (x, y). Information, that is, coordinate information of the area 200 shown in FIG. 3 is calculated.
[0030]
In step S13 for outputting the calculation result to the image exaggeration processing unit, the coordinate information calculated in step S12 is output to the image exaggeration processing unit 103. Receiving the coordinate information, the image exaggeration processing unit 103 adds additional information so that the coordinates of the area 200 are clear, and the coordinates other than the area 200 are blurred. Based on the added additional information and all image information, the image data dividing unit 104 divides the image data according to the right, center, and left display screens, and displays the image data on the respective display screens.
[0031]
In the method of exaggerating the image, in the area 200 when the image data is displayed, all the scanning lines are used and the luminance is increased. In areas other than the area 200, the number of scanning lines used is reduced by using every other scanning line, and the luminance is reduced to about one half.
Note that this can be realized by changing only one of the image data of the area 200 or the image data other than the area 200.
[0032]
FIG. 7 is a diagram showing an image exaggeration processing screen according to an embodiment of the present invention.
FIG. 7A is a diagram showing a range within a predetermined radius R from the line-of-sight position of the operator.
The area 200 indicating the range within the predetermined radius R is calculated by the CPU 114, and the calculation result is output to the exaggeration processing unit 103.
[0033]
FIG. 7B is a diagram illustrating a method of exaggeration processing.
Based on the calculation result, the exaggeration processing unit 103 performs processing such as adding information for clearly displaying the image data at each coordinate point of the area 200. On the other hand, in areas other than the area 200, the image data is changed by deleting data at each coordinate point so that the image data is not displayed every other line.
[0034]
FIG. 7C is a diagram illustrating a display example of an image subjected to the exaggeration process.
In the area 200, an image is displayed clearly and clearly, and in other areas, a blurred image is displayed.
FIG. 8 is a flowchart of the sound exaggeration process according to an embodiment of the present invention.
As shown in FIG. 8, the voice exaggeration processing includes step S21 for detecting the operator's line-of-sight position, step S22 for calculating coordinates (x, y), step S23 for calculating the position of sound from the calculation results, and coordinates (x, Step S24 extracts and exaggerates the sound data corresponding to y), and step S25 controls the sound data other than the coordinates (x, y) to a low sound.
[0035]
In step S <b> 21 for detecting the operator's line-of-sight position, the CPU 114 controls to fetch information indicating the operator's line-of-sight position from the coordinate detection processing unit 113.
In step S22 for calculating coordinates (x, y), a coordinate position is calculated from the line-of-sight position information captured in step S11.
In step S23 for calculating the position of the sound from the calculation result, the position of the sound is calculated from the calculated coordinate position.
[0036]
In step S24 of extracting and exaggerating the audio data corresponding to the coordinates (x, y), the audio data corresponding to the coordinates (x, y) indicating the calculated line-of-sight position is extracted, and the audio is exaggerated. For example, information for increasing the sound pressure of the extracted voice data is added.
In step S25 in which the sound data other than the coordinates (x, y) is controlled to be a small sound, information for lowering the sound pressure is added to the sound data at positions other than the coordinates (x, y) indicating the line-of-sight position.
[0037]
The CPU 114 outputs all audio data to which the sound pressure information is added to the audio data dividing unit 109. The audio data dividing unit 109 divides the received audio data according to the screen display units for each of right, center, and left, and reproduces audio from the audio unit.
According to the present invention, it is possible to exaggerate a part of the screen and the sound in accordance with the line-of-sight position of the operator, thereby realizing a more realistic effect.
[0038]
【The invention's effect】
As described above, according to the present invention, the following effects can be realized.
According to the present invention, a virtual space closer to a human visual function or auditory function can be expressed by locally clarifying an image and increasing sound pressure in accordance with the line-of-sight position of an operator. Etc.
[0039]
In addition, according to the present invention, by controlling the amount of image and sound information according to the operator's line-of-sight position, the operator can acquire only the necessary information, and the fatigue on the operator's vision and hearing can be reduced. It has features such as
Furthermore, according to the present invention, there is no need to directly equip the operator's face and ears with a special device for experiencing the virtual space, so that the operator's physical fatigue can be reduced.
[Brief description of the drawings]
FIG. 1 is an external view of a conventional example.
FIG. 2 is a functional configuration diagram of an embodiment of the present invention.
FIG. 3 is a diagram showing an appearance of an embodiment of the present invention.
FIG. 4 is a top view of an embodiment of the present invention.
FIG. 5 is an external view showing the operation of one embodiment of the present invention.
FIG. 6 is a flowchart of image exaggeration processing according to an embodiment of the present invention.
FIG. 7 is a diagram illustrating image exaggeration processing according to an embodiment of the present invention.
FIG. 8 is a flowchart of audio exaggeration processing according to an embodiment of the present invention.
[Explanation of symbols]
100 Control unit 101 DVD (Digital Video Disc)
102 Image data decompression unit 103 Exaggeration processing unit 104 Image data division units 105, 106, 107 Data processing unit 108 Audio data decompression unit 109 Audio data division units 110, 111, 112 Audio data processing unit 113 Coordinate detection processing unit 113a Pointing device 113b Eye tracking camera 114 CPU
120 Display unit 121, 122, 123 Image display unit 140 Audio unit 141, 142, 143, 144, 145, 146 Audio reproduction unit 160 VR device main body

Claims (7)

In an information processing apparatus that processes and outputs an input image,
Right image display means for dividing the input image into a right screen, a center screen, and a left screen, and displaying the divided right image on the right screen; and a center image for displaying the divided center image on the center screen Display division means for displaying on the display means and the left image display means for displaying the divided left image on the left screen ;
Upper right audio reproduction means for dividing audio data corresponding to the divided right screen, center screen, and left screen, and reproducing audio in the upper right direction, and lower right for reproducing audio in the lower right screen Audio reproduction means, central upper audio reproduction means for reproducing audio in the upper direction of the central screen, lower central audio reproduction means for reproducing audio in the lower direction of the central screen, and upper left audio reproduction means for reproducing audio in the upper direction of the left screen When the voice dividing means in the lower left audio reproducing means for reproducing the sound of the left screen downward Ru is reproduced output,
A pointing device for designating a desired position among the input images displayed on the right screen, the center screen, and the left screen ;
An image exaggeration that uses all scanning lines within a predetermined radius range centered on the position coordinates specified by the pointing device and performs image data processing so as to increase luminance to exaggerate the image within the predetermined radius range Processing means ;
Sound position calculating means for calculating a sound position from the position coordinates;
Voice enhancement processing means for exaggerating the voice by adding information for increasing the sound pressure to the sound position in the voice data ;
The information dividing apparatus, wherein the display dividing unit divides the input image subjected to the image data processing and displays the divided image on the right image display unit, the left image display unit, and the center image display unit . Having detection means for detecting the position coordinates of the user's line of sight ;
The information processing apparatus according to claim 1, wherein the image exaggeration processing unit performs the image data processing based on position coordinates designated by the pointing device or line-of-sight position coordinates detected by the detection unit. . 3. The information processing apparatus according to claim 1, wherein the image exaggeration processing unit performs the image data processing so as to thin out scanning lines to be displayed for each predetermined line outside the range of the predetermined radius. The voice exaggeration processing means, the information processing apparatus according to any one of claims 1 to 3, wherein the adding information to lower the sound pressure outside the sound position. In an information processing method for processing and outputting an input image,
A right image display procedure for dividing the input image into a right screen, a central screen, and a left screen, and displaying the divided right image on the right screen, and a central image for displaying the divided central image on the central screen A display division procedure for displaying in a display procedure and a left image display procedure for displaying the divided left image on the left screen ;
Upper right audio playback procedure for dividing the audio data corresponding to the divided right screen, center screen, and left screen, and playing the audio in the upper right direction, and the lower right for playing the audio in the lower right screen Audio playback procedure, upper center audio playback procedure for playing audio in the upper direction of the central screen, lower central audio playback procedure for playing audio in the lower direction of the central screen, and upper left audio playback procedure for playing audio in the upper direction of the left screen When the audio division procedure on the lower left audio reproduction procedure of reproducing sound on the left side of the screen under the direction Ru is reproduced output,
A position designation procedure for designating a desired position among the input images displayed on the right screen, the center screen, and the left screen ;
In the range of the predetermined radius centered on the position coordinates specified in the position specifying procedure, the image data processing is performed so that all the scanning lines are used and the luminance is increased, and the image of the range of the predetermined radius is exaggerated. An image exaggeration processing procedure for dividing and displaying the input image that has been subjected to the image data processing in the display procedure ,
A sound position calculation procedure for calculating a sound position from the position coordinates;
Wherein possess a voice Zhang procedure exaggerate speech by adding information to strengthen the sound pressure in the sound position in the audio data,
The display splitting procedure, the image data processing by dividing the input image is performed, the right image display procedure, an information processing method comprising Rukoto to display the left image display procedure, and the center image display procedure . 6. The information processing method according to claim 5, wherein in the image exaggeration processing procedure, the image data processing is performed so as to thin out scanning lines to be displayed for each predetermined line outside the range of the predetermined radius. The information processing method according to claim 5 or 6, wherein the voice exaggeration processing procedure adds information for lowering sound pressure except at the sound position.

Images