JP5197316B2 - Camera system - Google Patents

Description

The present invention relates to a camera system which controls the camera, the camera system to control the camera image of each object image with a single imaging element from different directions in which the optical path is controlled by a plurality of prisms Related.

In recent years, there are an increasing number of scenes where a camera is used, such as a camera as an in-vehicle sensor, a camera for imaging the front, rear, and both sides of a vehicle.
As a conventional photographing unit used in such a camera, an optical system composed of a lens and a prism is arranged in front of an imaging lens, and light incident from this optical system is passed through an imaging lens to form a single unit. There is known an in-vehicle camera unit configured to project a plurality of fields of view simultaneously by forming an image on an imaging surface of an imaging element (see, for example, Patent Document 1).

  However, when a photographing unit such as Patent Document 1 is used, the imaging lens disposed between the prism and the imaging element is configured as a compound lens, and is enclosed in a holder and assembled into one lens. For this reason, it is difficult to shorten the space between the incident position of the prism and the light receiving surface of the image sensor, and thus it is difficult to reduce the size of the photographing unit itself.

In addition, as a conventional photographing unit of this type, there is a method that uses a fisheye lens having an angle of view of about 200 degrees, cuts out the required angle of view, and provides image information to the driver after distortion correction (for example, patents). (Refer to Reference 2.) However, when such a photographing unit is used, an image with a very large peripheral portion cannot be obtained due to the fish-eye lens, and even if correction is performed, it is visually recognized due to lack of peripheral information. A high quality image could not be obtained.
JP 2000-89301 A JP 2006-262447 A

As described above, the conventional photographing unit using the optical system has a problem that it is small and an image with high visibility cannot be obtained over a wide field of view.
The present invention has been made in view of the above circumstances, and has a large horizontal angle of view obtained by using a plurality of free-form surface prisms and a single image sensor, and distortion to the periphery. It acquires less panoramic images, and an object thereof is to provide a camera system the whole or to be displayed on the display device as easily viewable image for the user a portion of the panoramic image.

The present invention employs the following configuration in order to solve the above problems.
That is, according to one aspect of the present invention, the camera system of the present invention includes a single image sensor that converts an object image into an electrical signal at a predetermined period and a plurality of free-form surface prisms, and a plurality of different directions. An optical system that forms an image of the subject from each of the subject images on a predetermined region on the image sensor by each free-form surface prism, and a moving image generator that generates a first moving image based on an electrical signal output from the image sensor Means, image dividing means for dividing the first moving image generated by the moving image generating means into partial images corresponding to the respective areas, and the partial images divided by the image dividing means in a predetermined arrangement in an image synthesizing means for synthesizing the second moving image, and a display control means for displaying on the display device the second video image synthesized by the image synthesizing means, the free-form surface flop Is arranged on the entrance surface of the prism, an elliptical fixed aperture, the first surface being the entrance surface has a negative refractive power in the YZ plane, and the second surface being the exit surface. and the third surface having a reflecting action is characterized Rukoto that having a positive refractive power.

In the camera system of the present invention, it is preferable that the optical system is disposed immediately before the image sensor with respect to a subject.
In the camera system of the present invention, it is desirable that the free-form surface prisms are arranged so that the predetermined areas do not overlap.

In the camera system of the present invention, it is preferable that the image synthesizing unit generates a panoramic image as a second moving image by synthesizing the partial images divided by the image dividing unit .

In the camera system of the present invention, it is preferable that the camera unit is disposed on the vehicle so as to capture the outside of the vehicle .

Also, according to one aspect of the present invention, the camera system of the present invention, a single imaging device for converting an object image into an electrical signal at a predetermined period, is composed of a plurality of free-form surface prism, a plurality of different An optical system that forms a subject image from a direction on each of the free-form surface prisms in a predetermined area on the image sensor, and a moving image that generates a first moving image based on an electrical signal output from the image sensor Generating means, image dividing means for dividing the first moving image generated by the moving image generating means into partial images corresponding to the respective regions, and the partial images divided by the image dividing means being predetermined Image composition means for synthesizing the second moving image by arrangement, display control means for displaying the second moving image synthesized by the image composition means on a display device, and detecting the traveling direction of the vehicle That an angle signal obtaining means for obtaining an angle signal from the direction detecting device, e Bei the speed signal acquisition means for acquiring a speed signal from the speed detector for detecting speed of the vehicle, the image synthesizing means, wherein The partial images divided by the image dividing unit are combined to generate a panoramic image as a second moving image, the camera unit is arranged in the vehicle so as to capture the outside of the vehicle, and the display control unit A part of the frame of the second moving image is displayed on the display device and displayed on the display device in accordance with the traveling direction of the vehicle based on the angle signal acquired by the angle signal acquisition means. to move the frame portion of the second moving image, taken by the traveling direction and the speed signal acquisition means of the vehicle based on the angular signal obtained by the angle signal acquisition means Moving the portion of the frame rate to the second moving image to be displayed on the display device in response of the vehicle based on said speed signal and said.

  According to the present invention, the imaging unit is downsized, a panoramic image with a large horizontal angle of view and a small distortion is acquired, and the whole or part of the panoramic image is displayed as an image that is easy for the user to view. Can be displayed.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
1A and 1B are perspective views from different directions of the appearance of the photographing unit, and FIG. 2 shows an exploded state of the photographing unit shown in FIGS. 1A and 1B. FIG. 3A is an external view of the frame 1 incorporating a free-form curved prism, FIG. 3B is a view showing the external appearance of incorporating the free-form curved prisms 4, 5, and 6 into the frame 1, and FIG. FIG. 4B is a diagram showing the YZ plane of the free-form surface prisms 4, 5, 6, and FIG. 5 shows the outer shape of the free-form surface prisms 4, 5, 6. FIG. 6 is a perspective view showing the outer shape of the pre-stage prism 7, and FIG. 7 is a table showing the positive and negative refractive powers in each surface of each optical surface.

A through hole h is formed in the frame 1 at a central portion in the longitudinal direction from the top surface 1a to the bottom surface 1b, and the through holes are formed on the inclined side surfaces 1c and 1d located on both sides of the top surface 1a and the bottom surface 1b. Partition recesses e _{1 to} e _{3 that} are notched so as to communicate with h are provided. Then, in the position facing the bottom of the compartment recesses e ₁ to e _3, the seat becomes 2 Tsude pair _{t 1 ~t 6 (t 1,} t 2 a _set, at t 3, _{t 4} 1 _{set, t 5,} t ₆ in one set) is formed (FIG. 3A, see Fig. 4A). The holder 2 is for mounting the frame 1, and the cover body 3 is for accommodating the free curved prisms 4, 5, 6 together with the frame 1 inside the frame 1. is there. The cover body 3 is provided with window holes 3a to 3c opened on the first surfaces (incident surfaces) of the free-form curved prisms 4, 5, 6 and is fixed to the frame 1 by screws or adhesion. Cover windows or the like are appropriately provided in the window holes 3a to 3c of the cover body 3 in order to prevent intrusion of foreign matters. Range of possible visual field) and fixed apertures 12 to 14 (see FIG. 4B).

The free-form surface prism 4 (hereinafter sometimes referred to as a first optical element) has an angle θ _{1 with} its left visual field facing from the side (the outer surface of the window hole 3c that is a recess formed in the cover body 3) to the front. = A free-form surface prism that enables photographing within a range of 60 degrees (see FIGS. 3B and 9). As shown in FIGS. 4B and 5A and 5B, the first optical element 4 has a first surface (incident surface) 4a having a light transmission function, and the interior of the light. It has three optical surfaces, a second surface 4b (which is an exit surface and located on the exit side of the bottom wall 1b of the through hole h) 4b and a third surface 4c which has a light reflecting function, and has a reflecting and transmitting function, the second face 4b of, and the third surface 4c is both a lens function, easily and accurately positioned and arranged to conform to the seat t ₅ and t ₆ of the frame 1 at the end portions on both sides of said second surface 4b Steps k ₁ and k ₂ that can be formed are formed (FIGS. 3A and 4B).

The free-form surface prism 5 (hereinafter also referred to as a second optical element, and the appearance is the same as the free-form surface prism 4) is a free-form surface prism similar to the first optical element 4 (FIGS. 2 and 2). 5). The second optical element 5 has an axis of the imaging surface of the imaging element 15 in FIG. 4B so that the first surface, which is the incident surface, faces in the direction opposite to the first optical element 4. It rotates 180 degrees around and is arranged in parallel with the through-hole h, so that the right visual field can be imaged in the range of angle θ ₂ = 60 degrees from right to the front.

  The free-form surface prism 6 (hereinafter sometimes referred to as a third optical element) is a free-form surface prism having the same configuration as the first optical element 4 and the second optical element 5 (FIG. 2). FIG. 5). The third optical element 6 is arranged in the through hole h in parallel in the same direction as the first optical element 4 (see FIG. 3B), and has a triangular prism shape on the entrance side of the first surface 6a. Or the front | former stage prism 7 formed in trapezoid shape etc. is arrange | positioned (refer FIG. 4A).

6A and 6B, the light incident from the pre-stage prism 7 whose outer shape is specifically shown is imaged on the image sensor 15 through the third optical element 6 (see FIG. 4A). In other words, the third optical element 6 can shoot a field of view (front field of view) in the range of 30 degrees to the right and left sides with respect to the straight front and a total angle θ ₃ : 60 degrees (see FIG. 9). . The front stage prism 7 has a thickness W ₁ that is thicker than the thickness W of the third optical element 6 in order to take in more light in the field of view to be photographed ((A) in FIG. 5, ( A)).

  A heat radiating plate 18 is provided on the back surface of the image sensor 15 via a substrate (flexible printed circuit board or the like) 17, and a protective plate 19 (glass or the like) is provided on the front surface of the image sensor 15. The substrate 17, the heat radiating plate 18, and the protection plate 19 are arranged (packaged) so as to correspond to the through hole h of the frame 1 in the through opening of the holder 2 together with the imaging element 15. The imaging element 15 is appropriately insulated from the substrate 17 and the heat radiating plate 18 by disposing an insulating member such as a silicon sheet.

The first to third optical elements 4 to 6 used in the photographing unit of the camera according to the embodiment of the present invention are free-form surface prisms having the same shape and the same medium and having a refractive index larger than 1.31 times. Each optical surface is composed of a free-form surface with the YZ plane as the only symmetrical surface, and has a rotationally asymmetric surface shape that gives power to the light beam and corrects aberrations caused by decentration. In addition, one in which the first and third surfaces of the three surfaces are rotationally asymmetric surfaces can be applied. Here, the YZ plane is a plane as shown in FIG. 4B (here, the directions of all the prism optical elements are the same), and the first to third prism optical elements 4-6. By using, an imaging lens or the like is unnecessary, and a high-definition image can be taken with a wide angle of view. In FIG. 4B, l ₁ is the axial principal ray, and 0 is the center of the stop.

  In the free-form curved prism applied in the present embodiment, the first surfaces 4a to 6a are transmitted in the order in which the light beams pass from the light incident side, and the internal reflection is performed by the second surfaces 4b to 6b and the third surfaces 4c to 6c. Then, the light passes through the second surfaces 4b to 6b again and reaches the image pickup device 15. As shown in FIG. 7, the first surfaces 4a to 6a are the XZ planes shown in FIG. 4B. Has a positive refractive power, and has a negative refractive power in the YZ plane. The second surfaces 4b to 6b have a negative refractive power in the XZ plane, a positive refractive power in the YZ plane, and further, XZ in the third surfaces 4c to 6c. Both have a positive refractive power in the YZ plane.

  Therefore, in the prisms 4 to 6, the free curved surfaces used in the camera of the embodiment of the present invention are negative and positive in the YZ plane from the first surfaces 4a to 6a to the second surfaces 4b to 6b that are the exit surfaces. Positive and positive, and, when simplified, a combination of negative and positive refractive powers from the incident side, so a so-called retrofocus arrangement is obtained. Therefore, the back focus can be lengthened, the distance between the image pickup element 15 and the optical element (free-form curved prism) can be appropriately ensured, and an optical member such as an IR cut filter or an optical low-pass filter. It is also advantageous to deploy.

  Further, in the XZ plane, the first surfaces 4a to 6a and the second surfaces 4b to 6b correct the coma aberration by the positive refractive power and the negative refractive power, and the third surfaces 4c to 6c The second surfaces 4b to 6c (exit surfaces in this case) which are main reflecting surfaces having positive refracting power and having negative refracting power have an action of flattening the image surface.

  Further, according to such an arrangement of the optical surfaces, in the off-axis rays, the chief rays inclined inward with respect to the image plane by the positive refractive power of the third surfaces 4c to 6c are changed to the second surfaces 4b to 6b ( In this case, the negative refracting power of the exit surface) makes the light ray substantially perpendicular to the imaging surface, which contributes to satisfying the image side telecentric condition in the XZ plane.

  In the imaging unit configured as described above, the optical image of the left 60 ° field of view focused by the first optical element 4 is formed on the image sensor 15 and focused on the right side 60 through the second optical element 5. A light image with a field of 60 degrees is formed in parallel with the light image with a field of 60 degrees on the left in the imaging device 15, and further, a light image with a field of 60 degrees on the front focused through the front prism 7 and the third optical element 6 is obtained. The image sensor 15 forms images in parallel with these optical images. That is, three fields of view are imaged in parallel on the single image sensor 15. In the photographing unit having such a configuration, photographing in a range of 180 degrees from the left side to the front side and the right side is possible.

  In the embodiment of the present invention, the photographing unit that forms the field of view in the range of 60 degrees on the left side, 60 degrees on the right side, and 60 degrees on the front side has been described as an example. The field of view of each free-form curved prism 4, 5, 6 is not limited to the range of 60 degrees.

  Further, the photographing unit may be configured by only two of the first optical element 4 and the second optical element 5, and either the left side or the right side is connected to either the front view or the rear view. It is also possible to construct a photographing unit by incorporating another optical element so as to form an image. In such a configuration, since the number of installed optical elements can be reduced, the structure can be further simplified and the photographing unit itself can be further downsized.

  In addition, in order to be able to image a 360-degree field of view, it is possible to adopt a configuration in which two sets of imaging units having the configuration shown in the present embodiment are combined on the back side, as well as a configuration in which the imaging unit is further vertically integrated. It can also be adopted.

  The photographing unit provided with the present invention is not only suitable for being mounted on a vehicle and photographing an external visual field, but is also useful as a surveillance camera that is installed inside a vehicle and monitors the inside and outside of the vehicle. It can also be applied to a capsule endoscope or the like.

FIG. 8 is a diagram illustrating an example in which the camera having the photographing unit according to the first embodiment of the present invention is mounted on a vehicle.
As shown in FIG. 8, a camera U that can simultaneously shoot in both the left and right directions and the front can be attached to the center of the front grille of the vehicle and used as a monitor on both sides and the front. In such an example of use, safety on both sides is further increased when the vehicle is traveling.

  FIG. 9 is a diagram for explaining the outline of the photographing unit according to the first embodiment of the present invention, FIG. 10 is a diagram showing an example of the photographing range of the subject, and FIG. 11 is a diagram of FIG. It is a figure for demonstrating the positional relationship and image size of three images image | photographed with the image pick-up element 15 which concerns on the 1st Embodiment of this invention when image | photographing the light image (image) of a to-be-photographed object.

  As shown in FIG. 9, the photographing unit according to the first embodiment of the present invention includes a prism group 50 including the above-described free-form surface prisms 4, 5, 6 and the front stage prism 7, one imaging element 15, and the like. Is arranged.

  The image pickup element of a general camera has a rectangular shape. When shooting a landscape, the camera is arranged so that the long side of the screen is in the horizontal direction and the short side is in the vertical direction, and the object is directly opposed. On the other hand, the camera according to the first embodiment of the present invention is arranged so that the short side of the image sensor is in the horizontal direction of the subject, and the photographing unit of the camera is divided into three images as shown in FIG. Are formed as images stacked in the long side direction of the image sensor. An image 51a shown in FIG. 11 is an image captured at 60 degrees on the left visual field by the free curved surface prism 4, an image 51b is an image captured at 60 degrees on the right visual field by the free curved surface prism 5, and an image 51c is shown in FIG. These are images picked up by the free-form surface prism 6 and the front stage prism 7 with a central visual field of 60 degrees.

  The image shown in FIG. 10 is an example of a subject, and is an image of a subject when a camera installed in front of the vehicle body captures a range of 180 degrees to the left and right around the traveling direction. The photographed image obtained when the subject shown in FIG. 10 is imaged indicates that it is composed of the images 51a, 51b, 51c shown in FIG.

FIG. 12 is a diagram showing the configuration of the camera system of the present embodiment.
In FIG. 12, the camera system includes a camera unit 52 that is an imaging device, a camera controller 71, and a driving controller 72.

  The camera unit 52 includes the above-described free-form curved prisms 4, 5, 6, the pre-stage prism 7, the image sensor 15, a TG / SSG unit 60, a CDS / AGC / A / D unit 61, a digital signal processing unit 62, a serial (Serial) I / F 63 and communication I / F 74 are provided.

The camera controller 71 includes a CPU 53, an operation input unit 54, an image processing MPU 57, a memory 58, a communication I / F 75, and a communication I / F 77.
The driving controller 72 is connected to a display 55, a display controller 56, a GPS / navigator 65, a rotation angle sensor 66, a direction indicator 67, a speed sensor 68, and a communication I / F 78 via a bus 79.

Note that settings such as white balance, exposure correction, and shutter speed of the captured image are the same as those of a general on-vehicle camera, and details thereof are omitted.
The camera unit 52 can capture moving images and still images, and the CPU 53 of the camera controller 71 and the image processing MPU 57 are connected to the camera unit 52 via communication I / Fs 74 and 75. The image processing MPU 57 performs processing such as image inversion, rotation, mirror image, reduction, enlargement, distortion correction, and image pasting. Then, the image processed by the image processing MPU 57 is output to the display 55 used as a monitor device via the communication I / Fs 77 and 78 and the display controller 56.

  Further, the camera unit 52 includes a prism group 50 for forming a subject image as shown in FIG. 10, and an image sensor 15 made of a CMOS / CCD disposed at an image formation position by the prism group 50. The camera unit 52 generates the captured image of FIG. 11 from the signal output from the image sensor 15, and the camera controller 71 generates a display image from the captured image. The display image generated by the camera controller 71 is output to the display 55 of the driving controller 72.

The TG / SSG unit 60 of the camera unit 52 generates an imaging timing pulse and drives the image sensor 15. The CDS / AGC / AD 61 of the camera unit 52 performs correlated double sampling on the image signal output from the image sensor 15, adjusts the gain according to the signal amount, performs A / D conversion, and outputs the result to the digital signal processing 62. . In the digital signal processing 62, the input digital image signal is subjected to processing such as color signal interpolation, color correction, and white balance, and the color image data generated as photographed image data is output to the camera controller 71.

The serial I / F 63 controls each part of the imaging device unit 52 in accordance with commands such as exposure setting and gain adjustment from the CPU 53.
The photographed image data input to the camera controller 71 is displayed on the display 55 via the display controller 56 after performing image processing to be described later based on an image display method instruction from the CPU 53.

Next, image processing and processing for displaying an image on the display 55 will be described.
FIG. 11 described above represents an image formed on the image sensor 15 via the free-form surface prisms 4, 5, and 6. If the number of pixels of the image sensor 15 is 750 pixels × 480 pixels, FIG. As shown in FIG. 4, if divided into three, the number of pixels in each field of view is 250 pixels vertically and 480 pixels horizontally.

  On the other hand, the three-divided image captures a field of view of a total of 180 degrees including left 60 degrees 51a, right 60 degrees 51b, and center 60 degrees 51c, but an image of 250 pixels vertically and 480 pixels horizontally is left as it is. If the images are combined and displayed on the display 55, the vertical direction is not a problem, but 1440 pixels are required in the horizontal direction, and the liquid crystal display 55 having a size of WVGA (800 horizontal pixels × 480 vertical pixels) generally used for in-vehicle use. A panoramic image obtained by combining the three images cannot be displayed as it is.

Therefore, the image processing MPU 57 executes processing such as image reduction, enlargement, and clipping according to various display methods described later.
The image data transfer to the display controller 56 is an example using an in-vehicle LAN in FIG.

Here, each device connected to the in-vehicle LAN will be described.
The GPS / navigation 65 processes the map information captured by the CPU 53 through the in-vehicle LAN in the car navigation system based on the GPS positioning information by the image processing MPU 57 according to the image display format, and displays the image on the display 55.

  The rotation angle sensor 66 transfers information on the rotation angle of the steering wheel operated by the driver of the vehicle to the CPU 53. The image processing MPU 57 cuts out an image in the traveling direction corresponding to the steering angle of the wheel calculated from the rotation angle information from the panorama image, processes it according to the set display format of the image, and displays it on the display 55.

  The direction indicator 67 transfers the information of the direction indicator operated by the driver of the vehicle to the CPU 53, and the image processing MPU 57 sets the image after viewing the image of the visual field corresponding to the predetermined traveling direction specified by the information. The display controller 56 displays the image on the display 55.

The speed sensor 68 detects the speed of the vehicle.
Next, the flow from image capture, image processing, and image display after image processing will be described.
FIG. 13 is a flowchart showing the flow of the camera control processing of the present embodiment, FIG. 14 is a diagram showing functional blocks of the camera system of the present embodiment, and FIG. 15 shows the first frame memory. FIG. 16 is a diagram illustrating a configuration of a second frame memory, and FIG. 17 is a diagram illustrating a configuration of a panoramic image generation table.

As described above, the image formed on the image sensor 15 via the free-form surface prisms 4, 5, and 6 is captured as the image 51 in FIG.
First, in step S701, the imaging unit 701 captures an image, and in the next step S702, an image signal obtained by imaging is converted into a digital image signal by an ADC (A / D converter) 702. In step S703, the digital image signal is subjected to processing such as color signal interpolation, color correction, white balance, and gamma correction by the image generation unit 703. Then, in the next step S704, the first frame memory 704 is stored. Stored as color photographed image data.

At this time, the image stored in the first frame memory 704 stores the same color image data as the image formed on the image sensor 15, as shown in FIG.
Note that the camera unit 52 includes an exposure detection unit 717 and an exposure controller 718 as appropriate, similar to a general video camera, digital still camera, and the like, and shoots according to the exposure value calculated based on the luminance information from the image sensor 15. The exposure is adjusted automatically.

  Next, in step S705, the captured image data stored in the first frame memory 704 is read by the read / write controller 705 with reference to the panorama image generation address conversion table stored in the address memory 706. And is stored in the second frame memory 707 with reference to the address conversion table (see FIG. 17). In the example shown in FIGS. 15 and 16, from the addresses (1, 1) to (1, 750) of the first frame memory 704, that is, from the addresses (1, 1) corresponding to 1A in FIG. 250) The image data read in the order of the addresses (1, 251) to (1, 500) corresponding to 1B and the addresses (1, 501) to (1, 750) corresponding to 1C are the second. Is written from the address (250,1) to (1,1) of the frame memory 707, followed by the address (250,961) to (1,961), and then from the address (250,481) to (1). , 481). That is, the 1A line of FIG. 15 is written to 1A of FIG. 16, the 1B line of FIG. 15 is written to the 1B line of FIG. 16, and the 1BC line of FIG. 15 is written to the 1BC line of FIG. Such processing is performed on the entire captured image data stored in the first frame memory 704.

  As a result of this processing, as shown in FIG. 16, the image stored in the first frame memory 704 is vertically and horizontally reversed, and further, the images of the left visual field 60 degrees, the central visual field 60 degrees, and the right visual field 60 degrees are 180 degrees. Are arranged so as to become one image corresponding to the visual field of the image and stored in the second frame memory 707 as a panoramic image.

  Incidentally, the subject image formed by the free-form surface prism is generally geometrically distorted. Therefore, even if the photographed image data obtained by photographing as described above is divided into three, and each partial image is rotated or inverted and written to the second frame memory, the partial image is not modified as shown in FIG. It cannot be combined as one panoramic image. In step S706, the three partial images corresponding to the respective images formed by the three free-form surface prisms 4, 5, and 6 stored in the second frame memory 707 are geometrically corrected, and distortion is corrected. Remove.

18A and 18B are drawings for explaining the principle of distortion correction, and FIG. 19 is a drawing for explaining the interpolation processing. An example of an image obtained by photographing the rectangular mesh shown in FIG. 18A as a subject is FIG. The rectangular mesh is distorted due to geometric aberration caused by the free-form surface prism. When the coordinates of the lattice point (i, j) of the rectangular mesh in FIG. 18A are (X _{i, j} , Y _{i, j} ), the lattice point is captured by the imaging unit 701 and the captured image is displayed. Maps to coordinates (x _{i, j} , y _{i, j} ). Based on the set of (X _{i, j} , Y _{i, j} ) and (x _{i, j} , y _{i, j} ), the following formula 2 is set so that the square error ρ represented by the following formula 1 is minimized. And the coefficient of the correction polynomial of Equation 3 below

as well as

To decide. It has been determined

as well as

An image without distortion can be obtained from a distorted image by Expression 2 and Expression 3 in which is substituted.

For each pixel of the second frame memory, the coordinate of the address of the center of the pixel was determined to minimize the square error ρ.

as well as

The address of the mapping destination is calculated by the correction polynomial (Equation 2 and Equation 3) into which is substituted and stored in advance in the address memory 706 as a distortion correction address conversion table in the address memory 706, and the panoramic image data in step S705 After the generation process, each pixel of the image stored in the second frame memory 707 is read, and the mapping destination address is specified with reference to the distortion correction address conversion table stored in the address memory 706.

Even if the coordinates of the center of the pixel before mapping are integers, the coordinates mapped by the correction polynomials of Equation 2 and Equation 3 may not be integers. For example, the pixel _{P 22} of the correction polynomial (equations 2 and 3) of FIG. 19 (A) results of the mapping, the even mapped onto _P'22 of FIG. 19 (B), the center is an integer of pixels Q The position ₂₂ is shifted without being mapped. In that case, the surrounding pixel (P _{11 to} P ₃₃ ) together with the pixel of interest (P ₂₂ ) is mapped to the coordinates calculated by the above formulas 2 and 3, and the center coordinates and the pixels of the destination pixels are mapped. From the value, the pixel value of the pixel Q ₂₂ at the coordinate position whose center is an integer is calculated by interpolation. As the interpolation processing, a known technique such as bilinear interpolation or bicubic interpolation is used. In the case of bicubic interpolation, interpolation processing is performed using pixels further outside the pixels P _{11 to} P ₃₃ .

  That is, for each pixel of the second frame memory, the pixel and its surrounding pixels are mapped by Formula 2 and Formula 3, and interpolation processing is performed by the interpolation unit 719 from the mapped coordinates and pixel values, and after distortion correction Are stored in the first frame memory 704. After completing the distortion correction of the three partial images, the image data is transferred from the first frame memory 704 to the second frame memory 707 to generate a panoramic image without distortion.

  In the above description, the captured image data stored in the first frame memory 704 is divided into three parts, and each partial image is rotated and inverted, temporarily stored in the second frame memory 707, and stored in the second frame memory 707. Although the distortion correction is performed for each stored partial image, the captured image data stored in the first frame memory 704 is divided into three, and each partial image is rotated or inverted, and distortion correction is performed at the same time. You may do it. Further, the distortion correction processing may use the read controller 708, the interpolation unit 709, and the third frame memory 710.

  Next, the “display setting process” subroutine of step S707 is executed. In the present embodiment, the image can be displayed on the display 55 in at least two or more formats as described above.

FIG. 20 is a flowchart showing the flow of the subroutine “display setting process” of the camera control process, and FIG. 21 is stored in the address memory 706 when the central portion of the entire field of view (180 degrees) is enlarged and displayed. FIG. 22 is a diagram showing a configuration of an adaless conversion table for creating a display image, and FIG. 22 is a diagram showing a display example when the central portion of the visual field (range of about 52 degrees) is enlarged and displayed on the entire screen of the display 55. FIG. 23 is a diagram illustrating a configuration of an address conversion table for generating a display image when the entire field of view (180 degrees) is reduced and displayed as a panoramic image. FIG. 24 illustrates a panoramic image of the entire field of view. It is a figure which shows the example of a display at the time of reducing and displaying on the display 55. FIG.

  The flow shown in FIG. 20 shows the flow of processing for setting the field of view according to the steering angle of the wheel from the panoramic image of the subject obtained through the processing up to step S706 and displaying it on the display 55.

First, in step S710, a change in monitor display format setting is detected. That is, it is detected whether or not the display mode switch 714 has instructed switching of the display format.
If there is a change in the setting of the monitor display format (step S710: YES), the process proceeds to step S711, and the address conversion table corresponding to the display format stored in the address memory 706 is selected. On the other hand, if there is no change in the setting of the monitor display format (step S710: NO), the process proceeds to the next step S712 while keeping the selected address conversion table.

  In step S712, it is determined whether the display pattern is single-screen display. Here, in the present embodiment, in the case of single screen display, an image to be displayed is enlarged or reduced, and when it is not single screen display, it is displayed without being enlarged or reduced.

  When the display pattern is a single screen display (step S712: YES), in step S713, the sensor controller 720 detects the steering angle of the wheel from information of the rotation angle sensor 716 of the steering wheel. Then, the traveling direction of the vehicle is specified from the steering angle. As a method for detecting the traveling direction of the vehicle, several methods described later may be used.

  Next, in step S714, the read controller 708 selects an address conversion table corresponding to the current display format described in the address memory 706, and is stored in the second frame memory 707 corresponding to the traveling direction of the vehicle. An address for reading out pixels in the partial image from the panoramic image is calculated. For example, when the format for displaying one image on the entire screen of the display 55 is set, when the steering angle of the wheel is 0 degree, the address conversion table of FIG. 21 is selected, and the address conversion table is referred to. The range of the central portion of the panoramic image of FIG. 16 is enlarged by about 52 degrees so that it can be displayed on the entire screen of the display 55. That is, the pixels in the range surrounded by the addresses (1, 491) and (250, 960) are read out, and the read portion of the image corresponds to the display 55 with reference to the address conversion table in step S715. The image is enlarged by 1.92 times to a WVGA (480 × 800 pixel) size image.

  In general, when the enlargement ratio is not an integer, most of the read pixels are mapped to non-integer addresses by address conversion. Here, the non-integer address indicates that the center of the pixel is a non-integer address. In that case, together with the pixels read from the second frame memory 707, the pixels around the pixels are also read as a block (step S715), and the mapping destination is determined by the address and magnification rate of each pixel in the block read by the address controller 708. The address of the write destination (non-integer) is determined (step S716). Subsequently, the interpolation unit 709 calculates the pixel value of the pixel located at the integer address of the display determined in step S716 by interpolation from the pixel value of the pixel read by the block and the non-integer address of the mapping destination (step S716). S717). As the interpolation method, a known method such as bilinear interpolation or bicubic interpolation is used as described above.

  Next, in step S718, the pixel value calculated by the interpolation processing is stored at a set address in the third frame memory 710 by a writing controller (not shown).

In step S719, it is determined whether or not writing of all the pixels in the third frame memory 710 is completed. If not completed (step S719: NO), the process returns to step S714 and the subsequent steps are repeated. If so (step S719: YES), in step S720, the image data stored in the third frame memory 710 is transmitted to the display 55 by the display controller 711, and the display image is updated, as shown in FIG. A screen as shown is displayed on the display panel 712.

  The display controller 711 in FIG. 14 corresponds to the display controller 56 in FIG. 10, and the display panel 712 corresponds to the display 55. Further, the first frame memory 704, the second frame memory 707, the third frame memory 710, and the address memory 706 of FIG. 14 are secured in the memory 58 of FIG. 10, and the read / write controller 705 and the read controller of FIG. The functions of 708, interpolation unit 709, interpolation unit 719, and sensor controller 720 are executed by the CPU 53 in FIG.

  On the other hand, when the display format for displaying the entire field of view on the display 55 is set, and the number of pixels on the screen of the display 55 is WVGA (800 horizontal pixels × 480 vertical pixels), as described above, 750 × 480. The panoramic image with a total field of view of 180 degrees generated by combining the three screens obtained by the pixel image sensor 15 has a horizontal pixel count of 480 × 3 = 1440 and is stored in the second frame memory 707. The displayed panoramic image data cannot be displayed on the display 55 in the same size. Therefore, it is necessary to reduce the image. In this case, the reduction process is executed in steps S714 to S719 described above. In step S714, an address conversion table as shown in FIG. 23 is referred to, and in the interpolation step in step S717, even in the case of reduction as in enlargement, a certain pixel and pixels around the pixel are used to be bilinear. In step S720, the display controller 711 updates the display screen of the display 55, and a panoramic screen with a 180-degree field of view as shown in FIG. 24 is displayed on the display panel 712. To display.

  When only a part of the panoramic image, that is, a part of the field of view taken by the camera unit 52 is displayed on the display 55 as shown in FIG. 22, the field of view to be displayed is automatically set according to the traveling direction of the vehicle. It is also possible to switch.

  FIG. 25 is a diagram showing the configuration of the address conversion table when the central portion of the field of view is displayed at the same magnification (1 magnification) so that one pixel of the captured image corresponds to one pixel of the display 55. FIG. 26 is a diagram showing an address conversion table when a captured image of a field of view covering the front from 90 degrees on the left side is displayed at an equal magnification. FIG. 27 shows a photographed image of the field of view covering the front from 90 degrees on the right side. FIG. 28 is a diagram showing an address conversion table when displayed at a magnification, FIG. 28 is a diagram showing a display example of an image of the central visual field converted by the address conversion table of FIG. 25, and FIG. 29 is an address conversion table of FIG. FIG. 30 is a diagram showing a display example of a left visual field image converted by the table, and FIG. 30 is a diagram showing a display example of a right visual field image converted by the address conversion table of FIG.

  In step S713 described above, the read controller 708 detects the direction of the direction indicator switch 715 set by the driver from the direction indicator 67 instead of detecting the steering angle of the wheel from the information of the rotation angle sensor 716 of the steering wheel. It may be detected. In this case, in step S714, one of the address conversion tables shown in FIGS. 25, 26, and 27 stored in the address memory 706 according to the setting direction of the direction indicator switch is referred to, and in step 720, in step S713. A screen in the same direction as the detected setting direction of the direction indicator switch 715 is displayed on the display 55.

Also, instead of switching the screens as shown in FIGS.
It is also possible to change the display on the display 55 so that the continuous field of view changes from FIG. 28 to FIG. For the continuous change in the field of view, instead of detecting the steering wheel rotation angle operated by the driver with the rotation angle sensor 66 described above and calculating the steering angle of the wheel from the rotation angle, the rotation sensors 66 The steering angle of the wheel may be directly detected.

  Note that, when the field of view is changed in synchronization with the steering angle of the wheel, it is executed by the processing from step S713 to step S719. A function for calculating the read address of the image is provided so that the image is displayed on the display 55. Then, the processing from step S713 to step S719 is similarly executed according to the calculated image reading address.

  FIG. 31A illustrates the calculation of the read address and the write address of the address conversion table when the field of view of the display screen is changed according to the wheel steering angle α (the right direction is positive with respect to the front of the vehicle). FIG.

  In the following description, the number of pixels (vertical × horizontal) of the panoramic image stored in the second frame memory 707 is H × W, and the number of pixels on the display screen 55 (vertical × horizontal) is M × N. The angle of the entire visual field in the horizontal direction of the panoramic image is θ, and the horizontal angle of the display center with respect to the center of the panoramic image (front of the vehicle) is φ (the right direction is positive). Then, as shown in FIG. 31A, the panorama image is displayed on the screen of the display 55 so as to circumscribe the top and bottom. In this case, the number of pixels in the horizontal direction of the panoramic image displayed on the display 55 is A, the angle of the visual field corresponding to the pixel number A is η, and the position in the panoramic image at the left end of the displayed visual field is in units of pixels. Y, ξ in angle units, and Z in the panoramic image at the right end of the displayed field of view as Z.

From FIG. 31, the following equations 4 to 8 hold.
M / H = N / A (Formula 4)
W / A = θ / η (Formula 5)
ξ = φ + (θ−η) / 2 (Formula 6)
W / θ = Y / ξ (Formula 7)
Z = Y + A (Formula 8)
From these equations 4 to 8, addresses Y and Z in the horizontal direction of the pixels a, b, c, and d of the panoramic image in FIG. 31A are obtained from the following equations 9 and 10.

Accordingly, the pixel a (1, Y) of the panoramic image stored in the second frame memory 707 is mapped to the address (1, 1) of the display 55, and the pixel b (H, Y) is mapped to the address (M) of the display 55. , 1), pixel c (1, Z) is mapped to address (1, N) of display 55, and pixel d (H, Z) is mapped to address (M, N) of display 55. Further, the pixel e of the address (S, T) in the range surrounded by the pixels a, b, c, d of the panoramic image is the address (S · M / H, (T−Y) · M) of the display 55. / H). If this address is an integer address, it is displayed on the screen as it is. If it is a non-integer address, as described above, the pixel e and surrounding pixels are mapped onto the display screen, and the address and pixel on the screen are displayed. The pixel value of the integer address is calculated from the value by correction.

The wheel steering angle α is proportional to the visual field direction φ. That is, the following expression 11 is established.
φ = K · α (Formula 11)
Here, K is a proportionality constant. In order to make the direction of the visual field coincide with the steering angle of the wheel, K = 1 may be set. Further, the change range of α may be divided into a plurality of ranges, and φ may be a fixed value in each range. Further, the proportionality constant K in Equation 11 may be changed according to the speed of the vehicle. For example, the proportional constant K1 may be set in the range where the vehicle speed detected by the speed sensor 68 is low, and K2 greater than K1 may be set as the proportional constant in the high speed range.

  FIG. 31B is a diagram showing an example of an address conversion table when a part of a panoramic image is displayed on the entire screen of the display 55 with the angle twice as large as the steering angle (α degree) of the wheel as the center of the field of view. It is.

  In the address conversion table shown in FIG. 31B, the size of the panoramic image with 180 degrees left and right visual fields stored in the second frame memory is 250 pixels × 1440 pixels (H × W), and the display screen size is 480 pixels × 800. This is an address conversion table when the steering angle of the pixel (M × N) and wheels is α degrees, and the direction φ at the center of the display is twice the steering angle (φ = 2α). Note that the decimal points below the read address in FIG.

  FIG. 32 is a diagram showing a display example in which three screens are displayed in two stages, FIG. 33 is a diagram showing a configuration of an address memory when three screens are displayed in two stages, and FIG. It is a figure which shows the example of a display which made 2 steps | paragraphs display of 2 screens, and the map information containing GPS information.

  When it is determined in step S712 of FIG. 20 that the display pattern is not a single screen display (step S712: NO), the displayed image is not enlarged or reduced, and the panoramic image stored in the second frame memory 707 is not enlarged. A part is displayed on the display 55 as it is.

  That is, when the display pattern is not a single screen display (step S712: NO), for example, when three screens of the central visual field, the right visual field, and the left visual field as shown in FIG. 32 are displayed in two stages, in step S721, The read controller 708 detects the steering angle of the wheel from the information of the rotation angle sensor 716 of the steering wheel. As the direction detection method, the above-described several methods may be used.

  Next, in step S722, referring to an address conversion table (for example, FIG. 33) stored in the address memory 706, an address of a pixel to be read from the second frame memory 707 is set, and set from the frame memory 707 in step S722. The pixel at the specified address is read.

  Next, in step S724, the address of the third frame memory of the pixel read out in step S723 is specified with reference to the address conversion table in FIG. 33, and in step S725, the pixel read out in step S723 is determined as the first address. 3 is stored in the address specified in step 724 of the third frame memory 710.

  In step S726, it is determined whether or not writing of three (or two) full screens to the third frame memory 710 is completed. If not completed (step S726: NO), the process returns to step S722. If the process is repeated and completed (step S726: YES), display update processing is performed by the display controller 711 in step S720, and a screen as shown in FIG. 32 is displayed on the display panel 712.

  Instead of the screen shown in FIG. 32, a screen including map information as shown in FIG. 34 can be displayed. Here, in the screen as shown in FIG. 34, the upper and lower 5 pixels of the image of 250 pixels in the vertical direction and 480 pixels in the horizontal direction are deleted to form two columns of 240 pixels in the vertical direction, 80 pixels on the left side are deleted, and the horizontal side is 400 pixels left This is an example in which the map information of the above-mentioned car navigation system is displayed by displaying the visual field image and the left visual field image from which the right 80 pixels are deleted in the upper stage, the central vertical 240 pixels, and the structure 480 pixels in the lower stage.

  When GPS is instructed, the GPS image from the GPS 713 is read out and read out by the controller 708, and the reduced or enlarged interpolated image is stored in the third frame memory 710. Similarly, the horizontal and vertical start addresses for the driver monitor may be set to generate and output an image.

  As described above, each embodiment of the present invention has been described with reference to the drawings. However, a camera system to which the present invention is applied is limited to the above-described embodiments and the like as long as the function is executed. It goes without saying that a single device, a system composed of a plurality of devices, an integrated device, or a system that performs processing via a network such as a LAN may be used.

  It can also be realized by a system comprising a CPU, ROM or RAM memory connected to a bus, an input device, an output device, an external recording device, a medium drive device, a portable storage medium, and a network connection device. That is, a ROM or RAM memory, an external recording device, and a portable storage medium that record software program codes for realizing the systems of the above-described embodiments are supplied to the camera system, and the computer of the camera system is programmed. Needless to say, this can also be achieved by reading and executing the code.

  In this case, the program code itself read from the portable storage medium or the like realizes the novel function of the present invention, and the portable storage medium or the like on which the program code is recorded constitutes the present invention. .

  Examples of portable storage media for supplying the program code include flexible disks, hard disks, optical disks, magneto-optical disks, CD-ROMs, CD-Rs, DVD-ROMs, DVD-RAMs, magnetic tapes, and nonvolatile memories. Various storage media recorded through a network connection device (in other words, a communication line) such as a card, a ROM card, electronic mail or personal computer communication can be used.

  In addition, the functions of the above-described embodiments are realized by executing the program code read out on the memory by the computer, and the OS running on the computer is actually executed based on the instruction of the program code. The functions of the above-described embodiments are also realized by performing part or all of the process.

Furthermore, a program code read from a portable storage medium or a program (data) provided by a program provider is provided in a function expansion board inserted into a computer or a function expansion unit connected to a computer. After being written to the CPU, the CPU on the function expansion board or function expansion unit based on the instructions of the program code
Etc. perform part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

  That is, the present invention is not limited to the above-described embodiments and the like, and can take various configurations or shapes without departing from the gist of the present invention.

It is a perspective view from the direction from which each external appearance of an imaging unit differs. It is the figure which showed the decomposition | disassembly state of the imaging | photography unit shown to (A) and (B) of FIG. It is a figure showing the external appearance of the flame | frame 1 incorporating a free-form surface prism. FIG. 2 is a diagram illustrating an appearance in which free-form curved prisms 4, 5, and 6 are incorporated in a frame 1. It is the figure which showed the AA cross section in FIG. It is a figure showing the YZ plane of free-form surface prisms 4, 5, and 6. FIG. 6 is a perspective view showing the outer shape of free-form curved prisms 4, 5, and 6. FIG. 6 is a perspective view illustrating an outer shape of a front prism 7. It is the figure which made the positive / negative of the refractive power in each surface of each optical surface tabular. It is a figure which shows the example which mounted the camera which has the imaging | photography unit of the 1st Embodiment of this invention in the vehicle. It is a figure for demonstrating the outline | summary of the imaging | photography unit which concerns on the 1st Embodiment of this invention. It is a figure which shows the example of the imaging | photography range of a to-be-photographed object. It is a figure for demonstrating the size of the picked-up image and screen which concern on the 1st Embodiment of this invention when image | photographing the optical image (image) of the to-be-photographed object of FIG. It is a figure which shows the structure of the camera system of this invention. It is a flowchart which shows the flow of the camera control process to which this invention is applied. It is a figure which shows the functional block of the camera system to which this invention is applied. It is a figure which shows the structure of a 1st frame memory. It is a figure which shows the structure of a 2nd frame memory. It is a figure which shows the structure of a panoramic image generation table. It is drawing for demonstrating the principle of distortion correction, (A) is an example of the object for calculating a distortion correction parameter, (B) is image data obtained when (A) is imaged. It is an example. It is a figure for demonstrating an interpolation process. It is a flowchart which shows the flow of the subroutine "display setting process" of a camera control process. It is a figure which shows the structure of the addressless conversion table for the display image preparation memorize | stored in the address memory 706 in the case of expanding and displaying the center part of a visual field. It is a figure which shows the example of a display at the time of enlarging the center part (range of about 52 degree | times) of a visual field, and displaying on the whole screen of the display 55. FIG. It is a figure which shows the structure of the address conversion table for a display image generation in the case of reducing and displaying the whole visual field (180 degree | times) as a panoramic image. It is a figure which shows the example of a display at the time of reducing and displaying 3 screens. It is a figure which shows the structure of the address conversion table in the case of displaying the center part of a visual field by equal magnification (magnification rate 1 time) so that 1 pixel of a picked-up image may correspond to 1 pixel of the display 55. FIG. It is a figure which shows the address conversion table in the case of displaying the picked-up image of the visual field from 90 degree | times on the left side with equal magnification. It is a figure which shows the address conversion table in the case of displaying the picked-up image of the visual field from 90 degree | times on the right side with equal magnification. It is a figure which shows the example of a display of the image of the center visual field converted with the address conversion table of FIG. FIG. 27 is a diagram illustrating a display example of a left visual field image converted by the address conversion table of FIG. 26. It is a figure which shows the example of a display of the image of the right visual field converted with the address conversion table of FIG. It is a figure for demonstrating the calculation of the read address and write address of an address conversion table in the case of changing the visual field of a display screen according to the steering angle (alpha) of a wheel (it makes a right direction positive). It is a figure which shows the example of the address conversion table in the case of displaying an image on the whole display centering on the angle of 2 times the steering angle ((alpha) degree) of a wheel. It is a figure which shows the example of a display which made 3 screens into 2 steps | paragraphs. It is a figure which shows the structure of the address memory in the case of displaying 3 screens in 2 steps. It is a figure which shows the example of a display which displayed 2 screens of left-right visual field and map information on 2 steps | paragraphs.

Explanation of symbols

e _{1 to} e ₃ partition recesses h through-holes t _{1 to} t ₆ seat U camera 1 frame 1a top wall 1b bottom wall 1c inclined side wall 2 holder 3 cover body 3a to 3c window hole 4, 5, 6 free-form surface prism 6a First surface 7 Previous prism 15 Image sensor 17 Substrate (flexible printed circuit board, etc.)
18 Heat sink 19 Protection plate 50 Prism group 51, 51a, 51b, 51c Image 52 Camera unit 53 CPU
54 Operation Input Unit 55 Display 56 Display Controller 57 Image Processing MPU
58 Memory 60 TG / SSG part 61 CDS / AGC / A / D part 62 Digital signal processing part 63 Serial I / F
65 GPS / Navi 66 Rotation Angle Sensor 67 Direction Indicator 68 Speed Sensor 71 Camera Controller 72 Driving Controller 74 Communication I / F
75 Communication I / F
77 Communication I / F
78 Communication I / F
79 Bus 701 Imaging unit 702 ADC (A / D converter)
703 Image generation unit 704 First frame memory 705 Read / write controller 706 Address memory 707 Second frame memory 708 Read controller 709 Interpolation unit 710 Third frame memory 711 Display controller 712 Display panel 713 GPS
714 Display mode switch 715 Direction indicator switch 716 Rotation angle sensor 717 Exposure detector 718 Exposure controller 719 Interpolator 720 Sensor controller

Claims (8)

A single image sensor that converts a subject image into an electrical signal at a predetermined cycle;
An optical system composed of a plurality of free-form surface prisms, and each of the subject images from different directions is formed on a predetermined region on the image sensor by each free-form surface prism;
A moving image generating means for generating a first moving image based on an electric signal output from the image sensor;
Image dividing means for dividing the first moving image generated by the moving image generating means into partial images corresponding to the respective areas;
Image combining means for combining the partial image divided by the image dividing means with a second moving image in a predetermined arrangement;
Display control means for causing the display device to display the second moving image synthesized by the image synthesis means;
Equipped with a,
The free-form surface prism has an elliptical fixed diaphragm disposed on the incident surface of the prism, and the first surface, which is the incident surface in the YZ plane, has a negative refractive power and is an output surface. camera system according to claim Rukoto the third surface having a second surface and a reflecting action is having a positive refractive power. The camera system according to claim 1, wherein the image synthesizing unit synthesizes the partial images divided by the image dividing unit to generate a panoramic image as a second moving image. The camera system according to claim 2 , wherein the camera unit is disposed in the vehicle so as to image the outside of the vehicle. Wherein the optical system, a camera system according to claim 1, characterized in that each free-form surface prisms are arranged so that the predetermined area does not overlap. The camera system according to claim 4, wherein the image synthesizing unit synthesizes the partial images divided by the image dividing unit to generate a panoramic image as a second moving image. The camera system according to claim 1 , wherein the optical system is disposed immediately before the image sensor with respect to a subject. The camera system according to claim 6 , wherein the image synthesizing unit synthesizes the partial images divided by the image dividing unit to generate a panoramic image as a second moving image. A single image sensor that converts a subject image into an electrical signal at a predetermined cycle;
An optical system composed of a plurality of free-form surface prisms, and each of the subject images from different directions is formed on a predetermined region on the image sensor by each free-form surface prism;
A moving image generating means for generating a first moving image based on an electric signal output from the image sensor;
Image dividing means for dividing the first moving image generated by the moving image generating means into partial images corresponding to the respective areas;
Image combining means for combining the partial image divided by the image dividing means with a second moving image in a predetermined arrangement;
Display control means for causing the display device to display the second moving image synthesized by the image synthesis means;
Angle signal acquisition means for acquiring an angle signal from a direction detection device for detecting the traveling direction of the vehicle;
A speed signal acquisition means for acquiring a speed signal from the speed detector for detecting a speed of said vehicle,
Bei to give a,
The image synthesizing unit synthesizes the partial images divided by the image dividing unit to generate a panoramic image as a second moving image;
The camera unit is arranged in the vehicle so as to image the outside of the vehicle,
The display control means displays a part of the frame of the second moving image on the display device, and the display device corresponds to the traveling direction of the vehicle based on the angle signal acquired by the angle signal acquisition means. The frame portion of the second moving image to be displayed is moved, and the traveling direction of the vehicle based on the angle signal acquired by the angle signal acquisition means and the speed signal acquired by the speed signal acquisition means the display characteristics and to Luke camera system that moves portions of frame of the second moving image to be displayed on the device in response to the speed of the vehicle based.