JP4331492B2 - Image processing apparatus, image providing server apparatus, image processing method, and image processing program - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image processing device, an image providing server device, an image processing method, and an image processing program for converting a circular curved surface image captured by a fisheye lens into a plane image divided into blocks of a predetermined size and compressing and encoding the image. In particular, the present invention relates to an image processing apparatus, an image providing server apparatus, an image processing method, and an image processing program capable of quickly generating a compression-encoded plane image from a curved surface image captured by a fisheye lens.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, a system that converts a circular curved surface image captured using a fish-eye lens (including a wide-angle lens) into a planar image and performs compression encoding is known. For example, in Japanese Patent Application Laid-Open No. 11-119303, a coordinated image of a three-dimensional image acquired by a fisheye lens is converted into a two-dimensional image, whereby a captured image having distortion due to the lens characteristics of the fisheye lens is converted into a normal two-dimensional image. Correct it. The two-dimensional image is subjected to image compression processing such as MPEG (Moving Picture Experts Group) method to generate a compression-encoded planar image.
[0003]
[Patent Document 1]
JP 11-119303 A
[0004]
[Problems to be solved by the invention]
However, the above-described conventional technique has a problem that it takes time to generate a compression-encoded plane image, and is practically used when a compression-encoded moving image is distributed in real time on the Internet. It was difficult. That is, in the above-described conventional technology, the plane conversion process and the image compression process are performed independently. This plane conversion process performs a complicated calculation process of conversion from a polar coordinate system to an orthogonal coordinate system on an image. Since this is performed for each pixel, the amount of calculation is enormous, and it takes a lot of time to generate a compression-coded plane image.
[0005]
Accordingly, the present invention has been made to solve the above-described problems caused by the prior art, and image processing capable of quickly generating a compression-encoded plane image from a curved surface image captured by a fisheye lens. An object is to provide an apparatus, an image providing server apparatus, an image processing method, and an image processing program.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problems and achieve the object, the invention according to claim 1 is an image in which a circular curved surface image picked up by a fisheye lens is converted into a flat image divided into blocks of a predetermined size and compressed and encoded. A processing apparatus, wherein the circular curved surface image (Radius r) The block division (N × n pixels) Circumferential division by size Is converted into a planar image composed of a plurality of isosceles triangles (base n × height r pixels), and then the planar image of each isosceles triangle is divided into blocks (n × n pixels). And have free pixels Planar image converting means for converting to a planar image, and the image in each block converted to a planar image by the planar image converting means After the original pixels are expanded so that the original pixels are arranged at equal intervals in the rows having empty pixels, the empty pixels are linearly interpolated using the surrounding original pixels to obtain n × n pixels having no empty pixels. In-block image generating means for generating a block image When, Generated by the in-block image generating means Compression encoding means for compressing and encoding the image in each block.
[0007]
According to this invention, a circular curved surface image is divided into blocks while being circumferentially divided according to the size of the block division, and is converted into a plane image, and the image in each block converted into the plane image matches the size of the block. The image in each enlarged block is compression-encoded. Therefore, instead of performing complicated calculation processing such as conversion from polar coordinate system to Cartesian coordinate system, simple plane conversion processing (circumferential division processing and enlargement processing) using block division, which is preprocessing of compression coding, is performed. By doing so, it is possible to quickly generate a compression-encoded plane image from the curved surface image captured by the fisheye lens.
[0008]
The invention according to claim 2 is the invention according to claim 1, further comprising a plurality of the planar image conversion means, in-block enlargement means, and compression encoding means, respectively, wherein the plurality of planar image conversion means and in-block enlargement. Each of the means and the compression encoding means performs a conversion process, an enlargement process, and a compression encoding process by sharing a predetermined area in the curved surface image.
[0009]
According to the present invention, the predetermined area in the curved surface image is shared, and the conversion process, the enlargement process, and the compression encoding process are performed. Therefore, by performing parallel processing for each predetermined region in the curved surface image, it is possible to generate a plane image compressed and encoded from the curved surface image captured by the fisheye lens more quickly.
[0010]
According to a third aspect of the present invention, a circular curved surface image picked up by a fisheye lens is converted into a plane image divided into blocks of a predetermined size and is compressed and encoded, and the compressed and encoded plane image is converted into a network. An image providing server device provided to the client device via the circular curved surface image (Radius r) The block division (N × n pixels) Circumferential division by size Is converted into a planar image composed of a plurality of isosceles triangles (base n × height r pixels), and then the planar image of each isosceles triangle is divided into blocks (n × n pixels). And have free pixels Planar image converting means for converting to a planar image, and the image in each block converted to a planar image by the planar image converting means After the original pixels are expanded so that the original pixels are arranged at equal intervals in the rows having empty pixels, the empty pixels are linearly interpolated using the surrounding original pixels to obtain n × n pixels having no empty pixels. In-block image generating means for generating a block image When, Generated by the in-block image generating means Compression encoding means for compressing and encoding the image in each block, and image transmission means for transmitting the planar image compressed and encoded by the compression encoding means to the client device. Features.
[0011]
According to this invention, a circular curved surface image is divided into blocks while being circumferentially divided according to the size of the block division, and is converted into a plane image, and the image in each block converted into the plane image matches the size of the block. The image in each enlarged block is compression-encoded, and the compression-encoded plane image is transmitted to the client device. Therefore, instead of performing complicated calculation processing such as conversion from polar coordinate system to Cartesian coordinate system, simple plane conversion processing (circumferential division processing and enlargement processing) using block division, which is preprocessing of compression coding, is performed. By doing so, it becomes possible to quickly generate and distribute a plane image compressed and encoded from a curved surface image captured by a fisheye lens.
[0012]
The invention according to claim 4 is the invention according to claim 3, further comprising a plurality of the planar image conversion means, in-block enlargement means, and compression encoding means, and the plurality of planar image conversion means and in-block enlargement. Each of the means and the compression encoding means share a predetermined area in the curved surface image to perform conversion processing, enlargement processing, and compression encoding processing, and the image transmission means respectively compresses the compression code by each compression encoding means. The converted planar image is multiplexed and transmitted to the client device.
[0013]
According to the present invention, each predetermined region in the curved surface image is shared, and conversion processing, enlargement processing, and compression coding processing are performed, and the compression-coded planar images are multiplexed and transmitted to the client device. . Therefore, by performing parallel processing for each predetermined region in the curved surface image, it is possible to generate and distribute a planar image compressed and encoded from the curved surface image captured by the fisheye lens more quickly.
[0014]
The invention according to claim 5 is the invention according to claim 3 or 4, wherein a predetermined region in the planar image transmitted by the image transmitting means is selected from the client device and received, and the selected region is selected. The image processing apparatus further includes area enlargement means for enlarging an image in the received predetermined area, and the image transmission means transmits the image enlarged by the area enlargement means to the client device.
[0015]
According to the present invention, a predetermined area in the transmitted plane image is selected and accepted from the client apparatus, an image in the predetermined area received by the selection is enlarged, and the enlarged image is transmitted to the client apparatus. Send. Therefore, it is possible to quickly deliver an image of a desired area for each user as if the user is performing a zoom operation virtually.
[0016]
The invention according to claim 6 is an image processing method for converting a circular curved surface image picked up by a fisheye lens into a planar image divided into blocks of a predetermined size and compressing and encoding the circular curved surface image. (Radius r) The block division (N × n pixels) Circumferential division by size After converting to a plane image composed of a plurality of isosceles triangles (n × r pixels), the plane image of each isosceles triangle is divided into blocks (n × n pixels), so that original pixels and empty pixels are contained in the block. Have Planar image conversion process for converting to a planar image, and images in each block converted to a planar image by the planar image conversion process After the original pixels are expanded so that the original pixels are arranged at equal intervals in the rows having empty pixels, the empty pixels are linearly interpolated using the surrounding original pixels to obtain n × n pixels having no empty pixels. In-block image generation process for generating block images When, Generated by the in-block image generation process And a compression encoding step for compressing and encoding the image in each block.
[0017]
According to this invention, a circular curved surface image is divided into blocks while being circumferentially divided according to the size of the block division, and is converted into a plane image, and the image in each block converted into the plane image matches the size of the block. The image in each enlarged block is compression-encoded, and the compression-encoded plane image is transmitted to the client device. Therefore, instead of performing complicated calculation processing such as conversion from polar coordinate system to Cartesian coordinate system, simple plane conversion processing (circumferential division processing and enlargement processing) using block division, which is preprocessing of compression coding, is performed. By doing so, it becomes possible to quickly generate and distribute a plane image compressed and encoded from a curved surface image captured by a fisheye lens.
[0018]
The invention according to claim 7 is an image processing program for causing a computer to execute an image processing method for converting a circular curved surface image captured by a fisheye lens into a planar image divided into blocks of a predetermined size and compressing and encoding the image. The circular curved surface image (Radius r) The block division (N × n pixels) Circumferential division by size After converting to a plane image composed of a plurality of isosceles triangles (n × r pixels), the plane image of each isosceles triangle is divided into blocks (n × n pixels), so that original pixels and empty pixels are contained in the block. Have Planar image conversion procedure for converting to a planar image, and images in each block converted to a planar image by the planar image conversion procedure After the original pixels are expanded so that the original pixels are arranged at equal intervals in the rows having empty pixels, the empty pixels are linearly interpolated using the surrounding original pixels to obtain n × n pixels having no empty pixels. In-block image generation procedure for generating block images When, Generated by the intra-block image generation procedure And a compression encoding procedure for compressing and encoding an image in each block.
[0019]
According to this invention, a circular curved surface image is divided into blocks while being circumferentially divided according to the size of the block division, and is converted into a plane image, and the image in each block converted into the plane image matches the size of the block. The image in each enlarged block is compression-encoded, and the compression-encoded plane image is transmitted to the client device. Therefore, instead of performing complicated calculation processing such as conversion from polar coordinate system to Cartesian coordinate system, simple plane conversion processing (circumferential division processing and enlargement processing) using block division, which is preprocessing of compression coding, is performed. By doing so, it becomes possible to quickly generate and distribute a plane image compressed and encoded from a curved surface image captured by a fisheye lens.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Exemplary embodiments of an image processing apparatus, an image providing server apparatus, an image processing method, and an image processing program according to the present invention will be described below in detail with reference to the accompanying drawings. In the following, the outline and features of the image processing system according to the present embodiment will be described, and then the configuration and processing procedure of the encoder in this image processing system will be described. Subsequently, the image in this image processing system will be described. The configuration of the providing server device will be described, and finally various modified examples will be described as other embodiments.
[0021]
(Outline and features of image processing system)
First, the outline and features of the image processing system according to the present embodiment will be described with reference to FIG. FIG. 1 is a system configuration diagram showing the configuration of the image processing system according to the present embodiment. As shown in the figure, an image processing system according to the present embodiment includes an image providing server device 20 having a fisheye lens camera 2 and an encoder 10, and a plurality of client devices 30 having a Web browser, and a network 1 such as the Internet. Are configured to be communicable with each other.
[0022]
In such an image processing system, the fish-eye lens camera 2 is installed on the ceiling of a gallery venue where pictures to be viewed are arranged, and collects a wide imaging range (for example, 360 degrees) without operating the camera such as panning. Image. Note that the image captured by the fisheye lens camera 2 is a circular curved image having distortion due to the lens characteristics of the fisheye lens, as shown in FIG.
[0023]
The encoder 10 also converts an image of a circular curved surface captured by the fisheye lens camera 2 into a plane image divided into blocks of a predetermined size and performs compression encoding (for example, image compression encoding such as JPEG or MPEG). It is a processing device. Then, in response to a request from each client device 30, the image providing server device 20 provides the client device 30 with the planar image compressed and encoded by the encoder 10 via the network 1. The client device 10 is, for example, a personal computer, a workstation, a home game machine, an Internet TV, a PDA, or a mobile communication terminal such as a mobile phone or a PHS.
[0024]
As described above, the image processing system shown in FIG. 1 schematically converts a circular curved surface image picked up by the fisheye lens camera 2 into a planar image divided into blocks of a predetermined size and performs compression encoding. The compressed and encoded plane image is provided to the client device 10 via the network 1. This image processing system has a main feature in the compression encoding process by the encoder 10, and can quickly generate a compression-encoded plane image from the curved surface image captured by the fisheye lens camera 2.
[0025]
Here, the main feature will be briefly described. In the encoder 10, the circular curved surface image is divided into blocks while being circumferentially divided according to the size of the block division, and converted into a planar image (see FIG. 2B). The image in each block converted into an image is enlarged so as to match the size of the block (see FIG. 3C), and the image in each enlarged block is compression-encoded. In other words, instead of performing complex calculation processing such as conversion from polar coordinate system to Cartesian coordinate system, simple plane conversion processing (circumferential division processing and enlargement processing) using block division, which is preprocessing of compression coding, is performed. By doing so, as described above, it is possible to quickly generate a “compressed plane image” from a “curved image captured by a fisheye lens”.
[0026]
Furthermore, in the position detection system according to the present embodiment, in relation to the main feature described above, the image providing server device 20 receives and accepts a predetermined region in the transmitted planar image from the client device 30. The image in the predetermined area selected and accepted is enlarged and transmitted to the client device 30. In other words, this makes it possible to quickly distribute an image of a desired area for each user as if the user is performing a zoom operation virtually.
[0027]
(Encoder configuration)
Next, the configuration of the encoder 10 in the image processing system according to the present embodiment will be described with reference to FIGS. 2 is a block diagram showing the configuration of the encoder 10 shown in FIG. 1, and FIG. 3 is an explanatory diagram for explaining the concept of planar image conversion and intra-block enlargement. FIGS. These are explanatory drawings for explaining a specific example of enlargement in a block.
[0028]
As described above, the encoder 10 is a device that converts a circular curved surface image picked up by the fisheye lens camera 2 into a planar image divided into blocks of a predetermined size, and compresses and encodes the image, as shown in FIG. The plane image conversion unit 11, the intra-block enlargement unit 12, the compression encoding unit 13, and the image memory 14.
[0029]
Among these, the plane image conversion unit 11 is a processing unit that divides a circular curved surface image picked up by the fisheye lens camera 2 into a plane image by dividing the block according to the size of the block division and dividing the block. Specifically, a circular (for example, radius r) curved surface image as shown in FIG. 3A is divided into blocks while being circumferentially divided in units of 8 pixels, which is the size of block division (for example, 8 × 8 pixels). To divide. As a result, as shown in FIG. 5B, isosceles triangles having a base of 8 pixels and a height r (8 × 3 pixels in this case) are connected, and the block is divided into a size of 8 × 8 pixels. To generate a planar image.
[0030]
The intra-block enlarging unit 12 is a processing unit that enlarges an image in each block converted into a planar image by the planar image converting unit 11 so as to match the size of the block. Specifically, each of the isosceles triangle images shown in FIG. 3B is enlarged in the horizontal direction, and as shown in FIG. 3C, the height r (here, 8 × A plane image in which quadrangles of (3 pixels) are connected is generated.
[0031]
Here, the intra-block enlargement process is conceptually described using a specific example. As shown in FIG. 4A, the upper side size of the trapezoid is 8 pixels for the block A shown in FIG. By enlarging so that an image of 8 × 8 pixels is obtained. Further, for the block B shown in FIG. 3 (b), as shown in FIG. 4 (b), by expanding the trapezoid so that the upper side size and the lower side size are both 8 pixels, Get an image. The enlargement ratio of the intermediate layer is estimated using the enlargement ratio of the upper side (maximum enlargement ratio) and the enlargement ratio of the lower side (minimum enlargement ratio).
[0032]
Further, the intra-block enlargement process will be described with an image using pixels (pixels). When the intra-block enlargement process as shown in FIG. 5A is performed, first, as shown in FIG. In addition, by expanding (developing) the pixels of each pixel, an 8 × 8 pixel image including empty pixels due to the expansion is obtained. Then, by shaping and interpolating such empty pixels using surrounding original pixels, an 8 × 8 pixel image having no empty pixels is obtained.
[0033]
Returning to the description of FIG. 2, the compression encoding unit 13 is a processing unit that compresses and encodes an image in each block enlarged by the intra-block enlargement unit 12. Specifically, in accordance with an image compression coding method such as so-called JPEG or MPEG, a compression code is obtained by performing a series of processes such as discrete cosine transform (DCT), quantization, and entropy coding on the image of each block. To generate a planarized image.
[0034]
The image memory 14 is a storage unit (storage unit) that stores image data and programs necessary for various processes by the above-described units (planar image conversion unit 11, in-block enlargement unit 12, and compression encoding unit 13). The image captured by the fisheye lens camera 2, the image data being processed by the intra-block enlargement unit 12, and the plane image image data compression-encoded by the compression encoding unit 13 are stored. The compression-encoded plane image data is input to the image providing server device 20 and is finally stored in the image providing server device 20.
[0035]
Note that the encoder 10 described above is not only configured by a dedicated image processing device (corresponding to the “image processing device” recited in the claims), but also by a known personal computer, workstation, PHS terminal, It can also be realized by mounting the functions of the above-described units as hardware or software on an information processing apparatus such as a portable terminal, mobile communication terminal, or PDA.
[0036]
(Encoder processing procedure)
Next, a processing procedure (particularly, image processing characterized by planar image conversion and in-block enlargement) by the encoder 10 shown in FIG. 2 will be described with reference to FIG. FIG. 6 is a flowchart showing a procedure of image processing by the encoder 10.
[0037]
As shown in the figure, when a circular curved surface image (see FIG. 3A) captured by the fisheye lens camera 2 is input to the encoder 10 (Yes in step S601), the circular curved image is divided into blocks. The image is divided into blocks while being circumferentially divided according to the size and converted into a planar image (step S602). That is, as shown in FIG. 3B, a plane in which isosceles triangles having a base of 8 pixels and a height r (8 × 3 pixels in this case) are connected and divided into blocks of 8 × 8 pixels in size. Get a statue.
[0038]
Subsequently, the encoder 10 enlarges the image in each block converted into the plane image by the plane image conversion unit 11 so as to match the size of the block (step S603). That is, as shown in FIG. 3C, a planar image is obtained in which a quadrangle having a base of 8 pixels and a height r (here, 8 × 3 pixels) is connected.
[0039]
The encoder 10 compression-encodes the image in each block enlarged by the intra-block enlargement unit 12 (step S604), and stores the compression-encoded image in the image memory 14 (step S605). Therefore, through the above-described series of processing, a “compressed plane image” is quickly generated from the “curved surface image captured by the fisheye lens” and stored in the image memory 14.
[0040]
If the encoder 10 compresses and encodes a moving image, the processes in steps S601 to S605 described above are sequentially repeated as compression encoding processing of the moving image, so-called I picture, B picture, and P picture. A compressed encoded stream of a moving image is generated and stored in the image memory 14.
[0041]
(Configuration of image providing server device)
Next, the image providing server device 20 in the image processing system according to the present embodiment will be described with reference to FIGS. 7 and 8. FIG. 7 is a block diagram showing the configuration of the image providing server device 20 shown in FIG. 1, and FIG. 8 is a diagram showing a screen image of the client device 30 shown in FIG.
[0042]
As described above, the image providing server device 20 is a device that provides a planar image compressed and encoded by the encoder 10 to the client device 30 via the network 1 in response to a request from each client device 30. As shown in FIG. 7, the configuration includes an input unit 21, an output unit 22, an input / output control IF unit 23, a communication control IF unit 24, a storage unit 25, and a control unit 26.
[0043]
Among these, the input unit 21 is an input unit that receives input of various types of information, and includes a keyboard and a mouse. For example, the input unit 21 receives and inputs operation instructions to the fisheye lens camera 2 and the encoder 10. The output unit 22 is an output unit that outputs various types of information, and includes a monitor and a speaker. For example, the output unit 22 displays and outputs the operation state of the fisheye lens camera 2 and the encoder 20.
[0044]
The input / output control IF unit 23 is means for controlling the input / output of data by the input unit 21, the output unit 22, and the encoder 10. For example, the input / output control IF unit 23 controls the input of the plane image image data compressed and encoded from the encoder 10. . The communication control IF unit 24 is a unit that controls communication regarding screen operation instruction information and image data exchanged with the client device 30 via the network 1.
[0045]
The storage unit 25 is a storage unit (storage unit) that stores data (for example, each user's authentication information) and programs necessary for various processes by the control unit 26, and is particularly closely related to the present invention. Is provided with an image memory 25a in terms of functional concept. Specifically, the image memory 25a stores the compressed and encoded image data of the planar image input from the encoder 10 in association with the imaging time.
[0046]
The control unit 26 has an internal memory for storing a control program such as an OS (Operating System), a program (for example, an image transmission program or a user authentication program) that defines various processing procedures, and required data. These are processing units that execute various processes. Particularly, those closely related to the present invention include a screen operation receiving unit 26a and a screen creating unit 26b in terms of functions.
[0047]
The screen operation accepting unit 26a is a processing unit that accepts an operation on a screen transmitted and displayed to the client device 30 from the client device 30, and the screen creating unit 26b is an operation accepted by the screen operation accepting unit 26a. It is a processing unit that creates screen data transmitted and displayed to the client device 30 from image data stored in the image memory 25a in response to a request.
[0048]
Here, the screen operation reception and screen creation processing will be specifically described. As shown in FIG. 8, the screen transmitted and displayed on the client device 30 is an overall image (for example, FIG. 3C). "Whole image field" for displaying a whole or part of the image image shown), "Partial image field" for displaying a partial image selected therefrom, and accepting virtual camera operations such as zooming, panning or tilting. Zoom field ".
[0049]
In this screen, for example, when the user instructs to select a predetermined area of the “whole image field”, the image providing server apparatus 20 enlarges the image of the area instructed to be selected by interpolation or the like and transmits it to the client apparatus 30. Such an enlarged image is displayed in the “partial image field”. Further, for example, when the user instructs camera operation such as zooming, panning, and tilting of the “zoom field”, the image providing server device 20 creates an image corresponding to the camera operation instruction and transmits the image to the client device 30. An image corresponding to zoom, pan, or tilt is displayed in the “overall image field”.
[0050]
(Other embodiments)
The embodiments of the present invention have been described so far, but the present invention is not limited to the above-described embodiments, and can be applied in various different embodiments within the scope of the technical idea described in the claims. It may be implemented.
[0051]
For example, in the present embodiment, the case where the fisheye lens camera 2 is installed in the gallery venue has been described. However, the present invention is not limited to this, and a place where monitoring by an image of a store, a home, a hospital, or the like can be performed. In addition, the fisheye lens camera 2 can be similarly installed and the present invention can be applied to any place where appreciation with images can be performed such as a sightseeing spot or a gallery venue.
[0052]
Further, in the present embodiment, the case where the vacant pixel is shaped and interpolated using surrounding original pixels in the intra-block enlargement process has been described, but the present invention is not limited to this, and the discrete cosine transform (DCT) is performed. ), The surrounding DCT coefficients may be used to compensate for the vacancy. Furthermore, in this case, if the image degradation is small, the interpolation using the DCT coefficient itself is omitted (the DCT coefficient of the empty pixel is set to “0”), and the compression encoding process is speeded up. You may do it.
[0053]
Furthermore, in this embodiment, the case where the encoder 10 is provided with the single planar image conversion unit 11, the in-block enlargement unit 12, and the compression encoding unit 13 has been described, but the present invention is not limited to this. A plurality of planar image conversion units 11, an in-block enlargement unit 12, and a compression encoding unit 13 may be provided. That is, in this case, each of the plurality of planar image conversion units 11, the in-block enlargement unit 12, and the compression encoding unit 13 shares a predetermined area in the circular curved surface image captured by the fisheye lens camera 2, respectively. Conversion processing, enlargement processing, and compression encoding processing are performed, and the compression-encoded planar images are multiplexed and transmitted to the client apparatus 10. Therefore, by performing parallel processing for each predetermined region in the curved surface image, it is possible to generate and distribute a planar image compressed and encoded from the curved surface image captured by the fisheye lens more quickly.
[0054]
In addition, among the processes described in this embodiment, all or part of the processes described as being automatically performed can be manually performed, or the processes described as being manually performed All or a part of the above can be automatically performed by a known method. In addition, the processing procedure, control procedure, specific name, and information including various data and parameters shown in the above-described document and drawings can be arbitrarily changed unless otherwise specified.
[0055]
Further, each component of each illustrated apparatus is functionally conceptual, and does not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each device is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed or arbitrarily distributed in arbitrary units according to various loads or usage conditions. For example, the encoder 10 and the image providing server device 20 shown in FIG. 1 may be integrated. Further, all or any part of each processing function performed in each device may be realized by a CPU and a program analyzed and executed by the CPU, or may be realized as hardware by wired logic.
[0056]
Note that the image processing method described in the present embodiment (planar transformation processing using block division of compression encoding) is realized by executing a program prepared in advance on a computer such as a personal computer or a workstation. be able to. This program can be distributed via a network such as the Internet. The program can also be executed by being recorded on a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, an MO, and a DVD and being read from the recording medium by the computer.
[0057]
【The invention's effect】
As described above, according to the invention of claim 1, 6 or 7, the block division, which is a pre-processing of compression coding, is performed instead of performing a complicated calculation process of conversion from a polar coordinate system to an orthogonal coordinate system. By performing the simple plane conversion process (circumferential division process and enlargement process) that is used, it is possible to quickly generate a compression-encoded plane image from the curved surface image captured by the fisheye lens.
[0058]
Further, according to the invention of claim 2, by performing parallel processing for each predetermined region in the curved surface image, it is possible to more quickly generate a compression-encoded planar image from the curved surface image captured by the fisheye lens. become.
[0059]
According to the invention of claim 3, a simple plane conversion process using block division, which is a pre-processing of compression encoding, is performed instead of performing a complicated calculation process of conversion from a polar coordinate system to an orthogonal coordinate system. By performing the circumferential division process and the enlargement process), it is possible to quickly generate and distribute a planar image that has been compression-coded from the curved surface image captured by the fisheye lens.
[0060]
According to the invention of claim 4, by performing parallel processing for each predetermined region in the curved surface image, a plane image compressed and encoded from the curved surface image captured by the fisheye lens can be generated and distributed more quickly. Is possible.
[0061]
According to the invention of claim 5, it is possible to quickly distribute an image of a desired area for each user as if the user is performing a zoom operation virtually.
[Brief description of the drawings]
FIG. 1 is a system configuration diagram showing a configuration of an image processing system according to an embodiment.
FIG. 2 is a block diagram showing a configuration of an encoder shown in FIG.
FIG. 3 is an explanatory diagram for explaining the concept of planar image conversion and intra-block enlargement.
FIG. 4 is an explanatory diagram for describing a specific example of enlargement within a block;
FIG. 5 is an explanatory diagram for describing a specific example of enlargement within a block;
FIG. 6 is a flowchart showing a procedure of image processing by an encoder.
7 is a block diagram illustrating a configuration of the image providing server apparatus illustrated in FIG. 1;
8 is a diagram showing a screen image of the client device shown in FIG. 1. FIG.
[Explanation of symbols]
1 network
2 Fisheye lens camera
10 Encoder
11 Planar image converter
12 Block enlargement
13 Compression encoder
14 Image memory
20 Image providing server device
21 Input section
22 Output section
23 I / O control IF section
24 Communication control IF section
25 storage unit
25a Image memory
26 Control unit
26a Screen operation acceptance unit
26b Screen creation part
30 client devices

Claims (7)

An image processing device that converts a circular curved surface image captured by a fisheye lens into a plane image divided into blocks of a predetermined size and compresses and encodes the image.
The circular curved surface image (radius r) is converted into a planar image composed of a plurality of isosceles triangles (base n × height r pixels) by dividing the circumference according to the size of the block division (n × n pixels). A plane image conversion means for converting each plane image of the isosceles triangle into blocks (n × n pixels) to convert into a plane image having original pixels and empty pixels in the block ;
For the image in each block converted into a planar image by the planar image converting means, after expanding the original pixels so that the original pixels are arranged at equal intervals in the row having the empty pixels, the empty pixels are replaced with the surrounding original pixels. In-block image generation means for generating a block image of n × n pixels without empty pixels by performing linear interpolation using
Compression encoding means for compressing and encoding an image in each block generated by the intra-block image generation means ;
An image processing apparatus comprising: A plurality of the planar image conversion means, in-block enlargement means and compression encoding means, respectively,
The plurality of planar image conversion means, in-block enlargement means, and compression encoding means each perform a conversion process, an enlargement process, and a compression encoding process by sharing a predetermined area in the curved surface image. The image processing apparatus according to claim 1. An image providing that converts a circular curved surface image captured by a fisheye lens into a planar image divided into blocks of a predetermined size and compresses and encodes the compressed encoded planar image to a client device via a network A server device,
The circular curved surface image (radius r) is converted into a planar image composed of a plurality of isosceles triangles (base n × height r pixels) by dividing the circumference according to the size of the block division (n × n pixels). A plane image conversion means for converting each plane image of the isosceles triangle into blocks (n × n pixels) to convert into a plane image having original pixels and empty pixels in the block ;
For the image in each block converted into a planar image by the planar image converting means, after expanding the original pixels so that the original pixels are arranged at equal intervals in the row having the empty pixels, the empty pixels are replaced with the surrounding original pixels. In-block image generation means for generating a block image of n × n pixels without empty pixels by performing linear interpolation using
Compression encoding means for compressing and encoding an image in each block generated by the intra-block image generation means ;
Image transmitting means for transmitting the planar image compressed and encoded by the compression encoding means to the client device;
An image providing server device comprising: A plurality of the planar image conversion means, in-block enlargement means and compression encoding means, respectively,
The plurality of planar image conversion means, in-block enlargement means, and compression encoding means each share a predetermined area in the curved surface image, perform conversion processing, enlargement processing, and compression encoding processing,
4. The image providing server device according to claim 3, wherein the image transmitting unit multiplexes the planar images compressed and encoded by the respective compression encoding units and transmits the multiplexed images to the client device. A region enlargement unit for selecting and accepting a predetermined region in the planar image transmitted by the image transmission unit from the client device, and enlarging an image in the predetermined region received by the selection;
The image providing server apparatus according to claim 3, wherein the image transmission unit transmits the image enlarged by the region enlargement unit to the client device. An image processing method for converting a circular curved surface image picked up by a fisheye lens into a plane image divided into blocks of a predetermined size and compression-coding the image,
The circular curved surface image (radius r) is circumferentially divided according to the size of the block division (n × n pixels) to be converted into a planar image composed of a plurality of isosceles triangles (n × r pixels), and then each two A plane image conversion step of converting a plane image of an equilateral triangle into blocks (n × n pixels) to convert into a plane image having original pixels and empty pixels in the block ;
For the image in each block converted into a planar image by the planar image conversion step, after expanding the original pixels so that the original pixels are arranged at equal intervals in the row having the empty pixels, the empty pixels are replaced with the surrounding original pixels. An intra-block image generation step of generating an n × n pixel block image without empty pixels by performing linear interpolation using
A compression encoding step for compression encoding the image in each block generated by the in- block image generation step ;
An image processing method comprising: An image processing program for causing a computer to execute an image processing method for converting a circular curved surface image captured by a fisheye lens into a plane image divided into blocks of a predetermined size and compressing and encoding the image.
The circular curved surface image (radius r) is circumferentially divided according to the size of the block division (n × n pixels) to be converted into a planar image composed of a plurality of isosceles triangles (n × r pixels), and then each two Planar image conversion procedure for converting a plane image of an equilateral triangle into a plane image having original pixels and vacant pixels in the block by dividing the plane image into blocks (n × n pixels) ;
For an image in each block converted into a planar image by the planar image conversion procedure, after expanding the original pixels so that the original pixels are arranged at equal intervals in a row having empty pixels, the empty pixels are converted into surrounding original pixels. An intra-block image generation procedure for generating an n × n pixel block image without empty pixels by performing linear interpolation using
A compression encoding procedure for compressing and encoding an image in each block generated by the intra-block image generation procedure ;
An image processing program for causing a computer to execute.

Images