JP4875887B2 - Image composition system and image composition method - Google Patents

Description

  The present invention relates to an image composition system and an image composition method for compositing other images with a panoramic image.

  In recent years, with the enhancement of functions of terminals such as mobile phones and personal computers and the development of infrastructure, services using images obtained by imaging up to 360 degrees around are widely deployed. For example, a wide-angle moving image or still image obtained by imaging 360 degrees at the maximum is displayed on the display screen of the terminal, and is used as a surveillance image for crime prevention in a building, real estate information introducing the inside of a room, or the like. These images are often displayed as rectangular images and are called panoramic images.

  A panoramic image is obtained by imaging with an imaging means such as a digital camera using an omnidirectional or wide-angle curved lens or curved mirror. Various techniques for generating panoramic images using curved lenses and curved mirrors have been developed from both image processing techniques and optical system techniques.

  When a panoramic image is generated by an image processing technique, a circular (annular) image captured by a curved lens or a curved mirror is generated by developing the panoramic image. The image picked up by the curved lens or the curved mirror is a circular image based on the equidistant projection method or a partial projection image thereof (hereinafter referred to as an equidistant projection image), and this equidistant projection image is projected based on the central projection method. A panoramic image is generated by converting it into an image (hereinafter referred to as a central projection image), a projection image based on equirectangular projection (hereinafter referred to as an equirectangular image), or the like.

  When generating panoramic images using optical system technology, curved lenses and curved mirrors have been developed that can capture panoramic images such as central projection images and equirectangular images without image processing by a program. The panorama image is directly generated by the above.

  FIG. 10 is an explanatory diagram for explaining the above-mentioned equidistant projection image, center projection image, and equirectangular image. For convenience of explanation, a case will be described in which a sphere is virtually set and converted into a planar image by each projection method or projection method.

  A symbol P in FIGS. 10A and 10B is a diagram for explaining an equidistant projection image. For example, in the case of a fish-eye lens that is a kind of curved lens, assuming that an image seen through the curved surface of the fish-eye lens from the position where the fish-eye lens is placed is pasted on the curved surface as it is, an image of the three-dimensional space pasted on the curved surface is An image projected so as to be crushed and pasted on a plane in a two-dimensional space is a circular equidistant projection image P obtained from a fisheye lens. When an omnidirectional mirror is used, a circular equidistant projection image P obtained from the omnidirectional mirror is obtained by projecting an image in a three-dimensional space projected on the curved surface of the mirror so as to be crushed and pasted on the plane of the two-dimensional space. It becomes.

  Reference symbol P1 in FIG. 10A is an explanatory diagram for explaining the central projection image. The central projection image is an image that is copied from a spherical surface to a plane that is in contact with the sphere, and is obtained by extending a line connecting a point on the sphere and the center of the sphere to the plane and copying the spherical image onto the plane. The central projection image may be obtained indirectly by converting the captured circular equidistant projection image by image processing technology, or may be obtained directly from a curved lens or curved mirror by optical system technology. is there. In the case of the image processing technique, theoretically, the equidistant projection image is converted into a virtual spherical surface by inverse transformation (arrow b1 in FIG. 10A), and the virtual spherical surface is further converted by central projection ( Although it is obtained by the arrow c1) in FIG. 10A, in practice, it is directly converted from the equidistant projection image by a conversion formula, a correspondence table, etc. (arrow a1 in FIG. 10A).

  Symbol P2 in FIG. 10B is an explanatory diagram for explaining an equirectangular image based on equirectangular projection. An equirectangular image is an image obtained by converting virtual parallel meridians on a spherical surface into a rectangular coordinate system on a plane at equal intervals. An equirectangular image can also be obtained indirectly by converting a captured circular equidistant projection image by image processing technology, and can be obtained directly from a curved lens or curved mirror by an optical technology. There is. In the case of image processing, theoretically, the equidistant projection image is converted into a virtual spherical surface by inverse transformation (arrow b2 in FIG. 10B), and the virtual spherical surface is further converted based on equirectangular projection. Although it is obtained by converting (arrow c2 in FIG. 10B), actually, it is directly converted from the equidistant projection image by a conversion formula, a correspondence table, etc. (arrow a2 in FIG. 10B). .

Japanese Patent Application Laid-Open No. 2004-259542 discloses a conversion for converting a circular equidistant projection image on a first coordinate system (polar coordinate system) into an equirectangular image on a second coordinate system (orthogonal coordinate system) by image processing. A technique for displaying a panoramic image using the principle is disclosed. In this conversion principle, the coordinates (X, Y) on the first coordinate system (input circular image) corresponding to the coordinates (x, y) on the second coordinate system are calculated using a predetermined coordinate conversion formula. By mapping the pixel of the circular equidistant projection image at the calculated coordinate (X, Y) on the first coordinate system to the coordinate (x, y) on the second coordinate system, The equidistant projection image is converted into an equirectangular image P2.
JP 2001-331789 A

Patent Document 2 below discloses a technique for generating a panoramic image using an optical system technique.
JP 2000-131613 A

  However, a panoramic image generated using a curved mirror or curved lens has the following drawbacks. Among panoramic images obtained by using a curved lens or a curved mirror, an image such as the central projection image P1 (FIG. 10A) has a sense of reality, but the distortion (scale ratio) varies depending on the region, and the sense of distance. There was a problem that was not properly expressed. For example, the center projected image P1 in FIG. 10A will be described as an example. Since distortion (scale ratio) is different between the vicinity of the center and the vicinity of the outer periphery, the object near the center is expressed smaller, and the vicinity of the center is expressed more than the actual area. It feels far away.

  On the other hand, it is conceivable to give priority to a sense of distance and extract and display only a region where distortion is close to 100% (for example, only near the center of the equirectangular image P2). In this case, since the object is expressed at a scale of almost the same ratio as the actual size, the size of the object is appropriate and the sense of distance is realistic, but there is a problem that the realistic feeling is lacking because of the planar display.

  If the above problem is a panoramic image obtained using a curved lens or a curved mirror, the panoramic image obtained indirectly by conversion using an image processing technique can also be directly converted without conversion using an optical system technique. The obtained panoramic image, and the panoramic image in which these images are represented by computer graphics, are common problems in any panoramic image, and a wide solution is desired.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an image composition system and an image composition method that can synthesize an image that appropriately expresses the size and distance of an object using panoramic images and is full of realism. .

An image composition system according to the present invention generates a first image and a second image by converting an original image captured by an imaging device, and the first image that is a panoramic image in the image composition system that performs image composition. Means for extracting and arranging an area corresponding to the designated area of the second image in the designated area of the first image, wherein the designated area of the first image is located near the center of the first image. a region close to the as compared to other regions distortion 100% of the first image, and the region corresponding to the specified area of the second image distortion than the specified area of the first image In order to obtain a composite image that is less and has a distortion aberration closer to 100% than that of the designated area of the first image, and has a realistic image, the designated area of the first image includes the second area. The area corresponding to the specified area of the image is extracted With the location, in the designated area outside the area of the first image, wherein the strain is large state is maintained.
The image composition method of the present invention is a first image that is a panoramic image in an image composition method in which a first image and a second image are generated by converting an original image captured by an imaging device, and the images are combined. Means for extracting and arranging an area corresponding to the designated area of the second image in the designated area of the first image, wherein the designated area of the first image is located near the center of the first image. a region close to the as compared to other regions distortion 100% of the first image, and the region corresponding to the specified area of the second image distortion than the specified area of the first image Is a region where distortion is closer to 100% than the designated region of the first image, and a region corresponding to the designated region of the second image is extracted from the designated region of the first image. Arranged outside the designated area of the first image So, by holding the distortion is large state, characterized by a full composite image realism. The designated area and the designated area region corresponding to the even may in area specified by being previously internally set in the system and step, may be a region that is specified by the user.

According to the present invention, an area corresponding to the designated area of the second image and having less distortion than the designated area is extracted and superimposed on the designated area of the first image that is a panoramic image. By using the second image in the designated area, the object becomes a more realistic ratio and the sense of distance is more appropriately expressed. In the area other than the designated area, the first image is used as it is. Thus, it is possible to generate a composite image that can make use of the realistic sensation specific to a panoramic image. Here, the panoramic image may be obtained using a curved lens or a curved mirror, or may be drawn by computer graphics. More preferably, the second image is an image without distortion.
In addition, since the specified area is an area in which distortion is close to 100% compared to other areas in the entire first image, in the specified area, an area in which the distortion is close to 100% is further 100%. By using the other areas as they are, it is possible to make use of the realism, so that it is possible to suppress a sense of incongruity due to image composition.

The designated area of the first image is preferably located near the center of the first image. The vicinity of the center of the image is a position that is easily noticed when displayed. In the vicinity of the center where attention is likely to be drawn, the size and distance feeling of the object are appropriately expressed using the second image, and in the peripheral area, the first image is used to give a sense of realism that is the overall atmosphere. The effect of image composition is high.
In addition, it is preferable that the region corresponding to the designated region of the second image has less distortion than the designated region of the first image and the distortion is close to 100%. Image distortion can be objectively compared by distortion. By arranging an image of an area where the distortion is close to 100% in the designated area of the first image, the designated area can be an image with little distortion. Here, the distortion aberration means the distortion aberration of the entire region. For example, the average value of distortion aberration in each region is calculated, and the average values are compared.

  The first image is preferably a central projection image. Here, the central projection image may be an image obtained by converting a circular equidistant projection image obtained by using a curved lens or a curved mirror by a conversion formula or a conversion table, a curved lens that can directly capture the central projection image, or the like. An image obtained using a curved mirror may also be used. Since the central projection image has excellent realism, it is possible to generate a composite image with an appropriate sense of distance while taking advantage of the realism of the central projection image.

  In addition, since the central projection image has a characteristic that distortion is close to 100% from the periphery toward the center, it is preferable that the designated region is close to the center where attention is easily paid and distortion is close to 100%. In the vicinity of the center, since the region where the distortion is close to 100% is made closer to 100%, the uncomfortable feeling caused by the image composition can be suppressed. In addition, the peripheral part having a distortion aberration that is 100% farther from the center can be used as it is, and as a whole, there is less sense of incongruity, and the composition is excellent in distance and presence. An image can be generated.

  The second image is preferably an equirectangular image based on equirectangular projection. The equirectangular image has a region where distortion is close to 100%, and the sense of distance is properly expressed in this region. By arranging an area corresponding to the designated area of the equirectangular image in the designated area of the first image, it is possible to obtain a composite image with excellent sense of distance. Further, in this case, in the equirectangular image that is the second image, it is preferable that the region corresponding to the designated region is a predetermined region having a point near the center at which distortion is closest to 100%. This is because this area has the most appropriate sense of distance.

  In particular, it is preferable that the first image is a central projection image, the second image is an equirectangular image, and the designated region is near the center of the first image. As a result, the above-described effects are comprehensively exhibited, and a composite image with little sense of discomfort and excellent sense of reality and distance can be obtained.

  Further, the second image is an equidistant projection image based on an equirectangular projection, a stereoscopic projection image or a polar projection image based on a orthographic projection, or an orthographic image or an orthogonal projection image based on an orthographic projection. Or, it is preferably an equisolid angle projection image based on the Lambert equal product azimuth projection. These images have a region where distortion is close to 100%, and the sense of distance is appropriately expressed in this region. By arranging the areas corresponding to the designated areas of these images in the designated area of the first image, it is possible to obtain a composite image with an excellent sense of distance.

  Further, it is preferable that a blurring process is performed on a boundary between both images in a state where an area corresponding to the designated area of the second image is arranged in the designated area of the first image. According to this, the boundary between the first image and the second image becomes inconspicuous, and a natural composite image is obtained by integrating both images.

  According to the present invention, by using the second image in the designated area, the object becomes a more realistic ratio, the sense of distance is more appropriately expressed, and in the area other than the designated area, the first image By using as it is, it is possible to generate a composite image that can make use of the realistic sensation unique to a panoramic image obtained by using a curved lens or a curved mirror. For example, when used as real estate information that introduces the interior of a room, it is possible to optimize the sense of distance and prevent misunderstandings such as depth while giving a sense of presence as if it were in the room. In addition, when used as a surveillance image for crime prevention, it is possible to display the size and distance of the object properly in the designated area without losing the sense of reality as a whole, and to help accurately grasp the monitoring location. Can do.

  Hereinafter, an image composition system and an image composition method according to a first embodiment of the present invention will be described with reference to the drawings. In this embodiment, an omnidirectional circular equidistant projection image obtained by using a curved lens or a curved mirror is used as an original image, and a first image and a second image that are panoramic images are generated from the original image. A case where the first image and the second image are synthesized will be described as an example. The image composition method of the present invention corresponds to the processing steps of the following image composition system.

  The first image was a central projection image, and the second image was an equirectangular image based on equirectangular projection. This is an example, and the first image and the second image may be images based on other projection methods and projections. For example, as the second image, an equidistant projection image based on orthorectangular projection, a stereoscopic projection image or polar projection image based on orthographic projection, or an orthogonal projection image or orthogonal projection image based on orthographic projection, Alternatively, an equisolid angle projection image based on the Lambert equal product azimuth projection may be used.

  In addition, central projection images, equidistant projection images, stereoscopic projection images, polar projection images, orthographic projection images, orthogonal projection images, isosolid angle projection images, etc. can be converted without using a dedicated curved mirror or curved lens. Obtained directly. These images may be used for the first image and the second image, and are not necessarily generated from the same original image.

  The image composition system 1 of the present embodiment is realized by a computer system such as a personal computer. FIG. 1 is a schematic diagram of an image composition system 1 and its peripheral devices according to this embodiment. The image composition system 1 is connected to an imaging device 2 such as an omnidirectional camera or a wide angle camera and a display device 3 such as a liquid crystal display.

(Imaging device)
The image pickup apparatus 2 can pick up an image of 360 degrees at the maximum. For example, as shown in FIG. 2, the image pickup apparatus 2A including a curved lens 21 such as an omnidirectional lens or a wide-angle lens and an image pickup means 23, This is an imaging apparatus 2B including a curved mirror 22 such as an azimuth mirror or a wide-angle mirror and an imaging means 23. A fish-eye lens with a viewing angle of 180 degrees is also included in the wide-angle lens. In the present embodiment, the first image and the second image that are panoramic images are obtained by using the imaging device 2 including the curved lens 21 and the curved mirror 22. Specifically, a circular equidistant projection image captured by the imaging device 2 is used as an original image P, and the original image P is converted to generate a first image and a second image.

  FIG. 3 is a conceptual diagram showing the relationship between the curved lens 21 or the curved mirror 22 (hereinafter referred to as a curved lens / mirror) and an image captured by the imaging means 23 such as a digital camera using the curved lens / mirror. It is. If a curved lens / mirror is attached to the image pickup means 23 and placed above (in the z-axis direction) above the three-dimensional space of the x-axis, y-axis, and z-axis, it is attached to the inside or outside of the curved lens / mirror. The image captured by the image capturing unit 23 can be considered, and the image captured by the image capturing unit 23 is a circular shape obtained by projecting the outer peripheral circular curved surface of the curved lens / mirror onto a flat surface. It can be assumed that the distance projection image P is obtained. An image captured using an omnidirectional curved lens / mirror is a donut-shaped equidistant projection image P with a central blank, and an image captured using a wide-angle curved lens / mirror such as a fisheye lens is a circular shape without a central blank. Is the equidistant projection image P. The angle in the horizontal direction in the circular plane C having the circumference of the curved surface of the curved lens / mirror as the circumference is the angle θ, and the elevation angle from the center of the circular plane C (the angle formed by the circular plane C and the radiation from the center) is the angle. If φ and the distance from the center in the circular projection image CP are the distance r, the point (θ, φ) on the curved surface is projected onto the plane on the assumption that the size of the circular equidistant projection image P is constant. The distance r from the center of the equidistant projection image P depends on the curvature of the curved surface. In the present embodiment, this equidistant projection image P is used as the original image P.

(Image composition system)
FIG. 4 is a diagram schematically showing a hardware configuration of a computer system that implements the image composition system 1. The image composition system 1 includes a CPU, a memory, an input device interface, a display device interface, an external device interface, a network interface, and an AV interface, which are connected via an internal bus. The image synthesizing system 1 is connected to the imaging device 2 via an AV interface, and further, a display device 3 such as a liquid crystal display is connected via a display device interface, and an input device 4 such as a keyboard and a mouse via an input interface. However, the storage means 5 such as a hard disk or a media drive is connected via an external device interface, and a network such as the Internet is connected via a network interface.

  FIG. 5 is a diagram schematically illustrating a functional configuration of the image composition system 1. The image composition system 1 includes first image generation means 11, second image generation means 12, and image composition means 13. FIG. 6 is an explanatory diagram for explaining the principle of the present image composition system.

(First image generating means)
The first image generation means 11 generates a panoramic image P1 from an original image (equal distance projection projection image) P obtained from a curved lens or curved mirror. As the panorama image, a central projection image will be described as an example, but a panorama image based on another projection method may be used. Specifically, when an original image P obtained by a curved lens or a curved mirror is input to the image composition system S, the original image P is converted into a first image using a conversion formula or conversion table stored in advance. Conversion to P1 (arrow a1 in FIG. 6). For example, the coordinate system of the original image P is set as the polar coordinate (α, β) of the declination α and the radius β, the coordinate system of the first image P1 is set as the polar coordinate (α ′, β) of the declination α ′ and the radius β ′. If '), the conversion formula is given as the following formula 1.

  The conversion from the original image P to the central projection image P1 is not limited to the above. For example, the original image P which is an equidistant projection image is converted into a virtual spherical surface (arrow b1 in FIG. 6), and the virtual image The first image P1 may be generated by projecting the spherical surface to the center (arrow c1 in FIG. 6).

  The first image P1, which is a central projection image, has a feature that there is little distortion toward the center and distortion is close to 100%, the object near the center is small, and the object is displayed larger as it goes to the outer edge. . Due to this difference in scale, the illusion that the area near the center is felt far away, but on the other hand, the depth is felt and the sense of presence is overflowing.

(Second image generating means)
The second image generation means 12 generates a second image P2 from an original image (equal distance projection projection image) P obtained from a curved lens or curved mirror. Here, an equirectangular image will be described as an example of the second image. Specifically, when an original image P obtained by a curved lens or a curved mirror is input to the image composition system S, the original image P is converted into a second image obtained by central projection using a pre-stored conversion formula or conversion table. 2 is converted into an image P2 (arrow a1 in FIG. 6). For example, in the case of a fish-eye lens which is a kind of curved lens, this conversion is performed with the coordinate system of the original image P as the polar coordinate (α, β) of the declination α and the radius β, and the coordinate system of the second image P2 as the horizontal axis. Assuming an orthogonal coordinate system (θ, φ) of θ and the vertical axis φ, it is given by Equation 2.

  Further, for example, in the case of an omnidirectional mirror which is a kind of curved mirror, the coordinate system of the original image P is the polar coordinate (α, β) of the declination α and the radius β, and the coordinate system of the second image P2 is the horizontal axis. Assuming an orthogonal coordinate system (θ, φ) of θ and the vertical axis φ, it is given by the following mathematical formula 3.

  The conversion from the original image P to the second image P2 is not limited to the above. For example, the original image P which is an equidistant projection image is converted into a virtual spherical surface (arrow b2 in FIG. 6), and the virtual image A second spherical image P2 may be generated by converting a simple spherical surface by equirectangular projection (arrow c2 in FIG. 6).

  The region near the center of the second image P2, which is an equirectangular image, is characterized by being less distorted than the region near the center of the first image P1, which is a central projection image, and distortion is close to 100%. , The size and sense of distance of the object are properly expressed.

  The generated first image P1 and second image P2 are displayed on output means such as a liquid crystal display device connected to the image composition system S. FIG. 7 is a diagram showing a display screen of the output means on which the first image P1 and the second image P2 are displayed.

On this display screen, the user can set the designated area r1 by designating an arbitrary area of the first image P1 by operating the mouse or the like. The designated area r1 is displayed by showing a broken line frame or the like on the first image.

  In addition, the display screen includes an item that prompts the user to select whether to synchronize the designated area r1 designated in the first image P1 and the area r2 in the second image P2 corresponding to the designated area r1. It has been. When the user checks the items and selects to synchronize, the image composition system S sets an area r2 (hereinafter referred to as a corresponding area r2) corresponding to the input designated area r1 on the second image P2. To do. The corresponding area r2 is set by the following process. In FIG. 6, the corresponding region r2 on the second image is obtained by inversely transforming the designated region r1 of the first image P1 onto the spherical surface and further transforming it into the second image P2. If the size and aspect ratio of the corresponding area r2 are different from that of the designated area r1, the corresponding area r2 is enlarged or reduced in the vertical and / or horizontal directions as appropriate. The corresponding area r2 is displayed by showing a broken line frame or the like on the second image.

  When the user does not select the synchronization function, uncheck the synchronization item. When the synchronization function is not selected, the synchronization is canceled and the user can arbitrarily designate the corresponding area r2 of the second image.

(Image composition means)
The image synthesizing unit 13 has a function of extracting and arranging the corresponding region r2 of the second image P2 in the designated region r1 of the first image P1. An image P3 shown in FIGS. 6 and 7 is a synthesized image P3 generated by the image synthesizing unit 13. The composite image P3 is in a state where the corresponding region r2 of the second image P2 is superimposed on the designated region r1 of the first image P1.

  The generated composite image P3 is in a state where the corresponding region r2 of the second image with less distortion is arranged near the center of the first image P1 that is a panoramic image. Thus, while using the presence of the peripheral portion of the first image P1 as it is, the size of the object is adjusted in the corresponding region r2, and the illusion of distance feeling that the vicinity of the center can be seen far away due to the difference in scale is alleviated. .

  Here, it is preferable to compare the distortion amounts of the regions r1 and r2 by distortion. Here, the distortion aberration means the distortion aberration of the entire areas r1 and r2. For example, each region r1, r2 is compared using an average value of distortion. The following method is mentioned as a method for obtaining the average value of the distortion aberration. For example, the average value of distortion aberration calculated for all the pixels in the region may be used, or the average value of distortion aberration obtained only for the pixel at the node obtained by dividing the region by an arbitrary lattice.

  Here, in the first image P1, the designated region r1 is a region in which distortion is close to 100% compared to other regions (regions excluding the designated region r1), that is, near the center in the present embodiment. It is preferable that This is because even if the corresponding region r2 having a distortion aberration closer to 100% than that of the designated region r1 is disposed in the designated region r1, there is less discomfort.

  Further, in the second image P2, the corresponding region r2 is set to a region where distortion is closer to 100% than other regions (regions excluding the corresponding region r2), that is, in the present embodiment, near the center. Is preferred. This is because the sense of distance can be effectively optimized by setting such a region as the corresponding region r2.

  In addition, the designated region r1 is set near the center, and the corresponding region r2 is arranged there, thereby optimizing the sense of distance of the region that is easily noticed, and the peripheral region is exposed to the entire atmosphere by utilizing the first image as it is. A feeling can be given.

The overlapping position of the designated area r1 and the corresponding area r2 may be determined by the image composition system 1, or may be arbitrarily determined by the user. The composite image P3 is displayed on the display screen (FIG. 7), and the displayed region of the composite image P3 can be moved by the user's mouse operation or the like, and the user can arbitrarily adjust the overlay position. It is possible.

  Further, in the synthesized image P3, the size of the corresponding area r2 can be changed by a user's mouse operation or the like, and the image synthesis system 1 can change the size of the corresponding area r2 by the user. The image in the corresponding area r2 is enlarged / reduced. As a result, the user can adjust the size of the corresponding region r2 in the first image P1 while looking at the balance of the composite image P3.

  Furthermore, it is preferable that the image synthesizing unit 13 has a function of performing a blurring process on the boundary between the superimposed first image P1 and second image P2 in the synthesized image P3. The composite image P3 becomes an integrated image as a whole because the boundary between the first image P1 and the second image P2 is not noticeable. As a result, the synthesized image P3 is reduced in discomfort due to the synthesis and becomes a more realistic image. The blurring process is performed by averaging pixels near the boundary in this embodiment, but other methods may be used.

  When using the image composition system 1, first, the original image P imaged by the imaging device 2 is input to the image composition system 1. In the image composition system 1, the first image composition unit 11 generates a first image, the second image composition unit 12 generates a second image, and displays the images P1 and P2 on the display screen. When the initial state of the designated region r1 of the first image P1 and the corresponding region r2 of the second image P2 is set, the regions r1 and r2 are displayed on the images P1 and P2 on the display screen by broken lines or the like. . Next, the user designates the designated area r1 of the first image P1 by a mouse operation or the like. The designated area r1 is indicated by a broken line or the like in the first image P1. At that time, when the user checks a predetermined item, the image composition system 1 sets the corresponding region r2 corresponding to the designated region r2 in the second image P2, and displays it on the display screen by a broken line or the like. To do. When the user does not check the item, the corresponding area r2 is set by a mouse operation or the like and displayed on the display screen. Further, the image composition system 1 generates a composite image P3 by arranging the corresponding region r2 of the second image in the designated region r1 of the first image P1 by the image composition unit. When the user wants to perform blurring processing on the boundary between the region of the first image P1 and the corresponding region r2 of the second image in the composite image P3, check the item “blur boundary” to indicate the degree of blurring. Move the pointer to the scale to specify the degree of blur. When the item “blur boundary” is checked, the image composition system 1 performs the blurring process to the specified degree.

(Other embodiments)
The first image P1 and the second image P2 may be directly input to the image composition system 1. Each input image P1, P2 may be a panoramic image generated by converting the original image P, or the panoramic image was captured using a curved lens or curved mirror obtained directly without conversion. It may be an image.

  In this case, the image synthesizing system 1 according to the present embodiment does not require conversion processing from the original image P to the images P1 and P2, and thus the first image generating unit 11 and the second image generating unit 12 are provided. It is not necessary to prepare. Note that the image processing apparatus includes a determination unit that determines whether the input image is a circular equidistant projection image P, the first image P1, or the second image P2. The processing by the first image generation unit 11, the second image generation unit 12, and the image composition unit 13 may be activated according to P1 and P2.

  Further, as the second image P2, the second image is an equidistant projection image based on an equirectangular projection, a stereoscopic projection image or a polar projection image based on a orthographic projection, or an orthographic projection. An orthographic projection image or an orthogonal projection image based on this, or an equisolid angle projection image based on the Lambert equal product azimuth projection may be used. FIG. 8A is an example of a composite image P3 when the second image is a stereoscopic projection image or a polar projection image, and FIG. 8B is a view when the second image is an orthographic image or an orthogonal projection image. It is an example of the composite image P3, (c) is an example of the composite image P3 when it is an equisolid angle projection image, and (d) is an example of the composite image P3 when it is an equidistant projection image. In both cases, the first image is a central projection image. The corresponding region r2 of the second image has a distortion aberration close to 100% as compared with the designated region r1 of the first image, and the effect of the present invention is exhibited regardless of which image is used as the second image.

(Example)
An embodiment of the present invention will be described by taking as an example a case where an indoor landscape is provided as a panoramic image in the real estate field or the like. FIG. 9 is a diagram for explaining the present embodiment. A first image P1 is generated by the first image generation unit 11, and a second image P2 is generated by the second image generation unit 12. The image composition means 13 superimposes the corresponding region r2 of the second image P2 on the predetermined region r1 of the first image P1, blurs the boundary between the images P1 and P2, and the composite image P3 is displayed. Generated. The composite image P3 is displayed on the screen of the display device 3. The user has a simulated experience in the room by viewing the composite image P3 displayed on the display device 3. The composite image P3 is full of realism, and the size and distance of the object are realistic in the area near the center. The user does not misunderstand the size of an object such as a table, the depth of the room, etc. while feeling as if he / she is in the room. These images may be taken with a curved lens or a curved mirror, or may be CG.

  The image synthesizing system of the above embodiment has been described by way of example of a computer system such as a personal computer, but may be realized by a mobile terminal such as a digital camera or a mobile phone. Each of the above means and methods may be realized by a program or a circuit such as an IC chip. The image composition system 1, the imaging device 2, and the display device 3 may be connected via a network.

  In the above-described embodiments and examples, a still image has been described as an example. However, for a moving image captured by a moving image capturing unit such as a CCD camera, the same processing is performed on each frame constituting the moving image. This is applicable to moving images.

Schematic diagram of the image composition system of this embodiment and its peripheral devices (A) is a schematic diagram of an imaging device provided with a curved lens, (b) is a schematic diagram of an imaging device provided with a curved mirror. Conceptual diagram showing the relationship between a curved lens / mirror and a projected image taken by a digital camera using the curved lens / mirror The figure which showed typically the hardware constitutions of the computer system which actualizes the image composition system Diagram showing functional configuration of image composition system Explanatory diagram for conceptually explaining the principle of image composition of the present invention The figure which shows the display screen on which a 1st image, a 2nd image, a synthesized image, an operation button, etc. are displayed. (A) is an example of the composite image P3 when the second image is a stereoscopic projection image or polar projection image, and (b) is the composite image P3 when the second image is an orthographic image or an orthogonal projection image. Example, (c) is an example of a composite image P3 when it is an equisolid angle projection image, and (d) is an example of a composite image P3 when it is an equidistance projection image The figure which shows the Example of the display screen of the image composition system of this invention Explanatory diagram for conceptually explaining equidistant projection image, central projection image, and equirectangular image

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image composition system 11 1st image generation means 12 2nd image generation means 13 Image composition means 2 Imaging device 21 Curved lens 22 Curved surface mirror 23 Digital camera 3 Display device C Circular plane CP Circular projection image P Original image P1 1st Image P2 second image P3 composite image r1 designated region r2 corresponding region

Claims (4)

In an image composition system that generates a first image and a second image by converting an original image captured by an imaging device, and performs image composition,
Means for extracting and arranging an area corresponding to the designated area of the second image in a designated area of the first image that is a panoramic image, wherein the designated area of the first image is the first image distortion as compared to other areas of and located near the center of the first image is a region close to 100%, and the area corresponding to the specified area of the second image, the first A region that is less distorted than the designated region of the image and has a distortion aberration closer to 100% than the designated region of the first image,
In order to obtain a composite image overflowing with a sense of presence, an area corresponding to the designated area of the second image is extracted and arranged in the designated area of the first image, and the designated image of the first image is designated. An image composition system characterized in that a state of large distortion is maintained in an out-of-region region . The first image is a central projection image, and the second image is an equirectangular image based on an equirectangular projection, an equidistant projection image based on an equirectangular projection, or a plane azimuth. stereographic projection images or pole projected image based on the projection or orthographic projection image or orthogonal projection image based on the orthographic azimuthal projection, or claim 1, characterized in that the steric angle projection image based on the Lambert azimuthal The image composition system described. In an image composition method for generating a first image and a second image by converting an original image captured by an imaging device, and performing image composition,
Means for extracting and arranging an area corresponding to the designated area of the second image in a designated area of the first image that is a panoramic image, wherein the designated area of the first image is the first image distortion as compared to other areas of and located near the center of the first image is a region close to 100%, and the area corresponding to the specified area of the second image, the first A region that is less distorted than the designated region of the image and has a distortion aberration closer to 100% than the designated region of the first image,
In the designated area of the first image, an area corresponding to the designated area of the second image is extracted and arranged, and the area outside the designated area of the first image keeps a large distortion state. Thus, an image composition method characterized by creating a composite image full of realism . The first image is a central projection image , and the second image is an equirectangular image based on equirectangular projection, an equidistant projection image based on equirectangular projection, or a projecting azimuth projection stereographic projection images or pole projected image based on, or, the orthogonal projection image or orthogonal projection image based on the orthographic azimuthal projection, or, according to claim 3, characterized in that the steric angle projection image based on the Lambert azimuthal Image composition method.

Images