JP7182953B2 - Information processing device, system, information processing method and program - Google Patents

Description

The present invention relates to an information processing device, system, information processing method, and program.

2. Description of the Related Art Conventionally, a fish-eye camera equipped with a fish-eye lens has become popular as one type of surveillance camera. A single unit can perform 360° monitoring, which leads to a reduction in the number of units and costs. Further, there are cameras capable of outputting not only an omnidirectional image obtained from a fisheye camera as it is, but also an image or video that has undergone multiple types of cropping processing and distortion correction processing. With such a camera, when various video content analyses, such as human body detection and moving body detection, are used, images after clipping processing and distortion correction processing are often used. Furthermore, such a camera can output a plurality of images that have undergone a plurality of types of clipping processing and distortion correction processing, and can monitor according to the imaging environment and installation environment.
However, it is necessary for the user to set an appropriate clipping process or distortion correction method for the scene being monitored by the user via the user interface. In addition, if the clipping process and distortion correction method that have been set once are used, changes in the imaging scene cannot be handled.
For example, Patent Literature 1 discloses a method of detecting feature amounts of a plurality of images and switching to a display device for each designated continuous display time of each image.

JP 2015-222917 A

However, regarding the display of captured images obtained from a fisheye camera, there is a growing demand for a display suitable for a captured scene or the like.

An information processing apparatus according to the present invention provides a pattern for outputting a partial image obtained from an omnidirectional image based on the number and positions of regions of interest obtained from the omnidirectional image, which is an image captured using a fisheye lens. a setting means for setting the , a cutting means for cutting a partial image including the attention area from the omnidirectional image based on the pattern set by the setting means, and a distortion for the partial image cut by the cutting means correction means for performing correction processing; and output means for outputting a partial image subjected to distortion correction processing by the correction means in accordance with the pattern set by the setting means, wherein the setting means further includes the fisheye lens. The pattern is set based on the imaging direction of the imaging unit, one partial image is cut when the imaging direction is horizontal, and two partial images are cut when the imaging direction is vertical. Set the pattern to cut out .

According to the present invention, it is possible to output an image suitable for an imaging scene or the like.

It is a figure which shows an example of the hardware constitutions of a camera. It is a figure which shows an example of an imaging part. It is a figure which shows an example of the software configuration of a camera. It is a figure which shows an example of a captured image. It is a figure which shows an example of the image output by an output part. It is a figure which shows an example of a captured image. It is a figure which shows an example of the image output by an output part. 4 is a flow chart showing an example of information processing of a camera; 4 is a flow chart showing an example of information processing of a camera; FIG. 10 is a diagram showing an example of pattern 1; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 2; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 3; It is a figure which shows an example of the image on which the distortion correction process was performed. It is a figure which shows an example of the image output by an output part. It is a figure which shows an example of the hardware constitutions of the apparatus which comprises an imaging system. FIG. 2 is a diagram showing an example of a software configuration of devices that constitute an imaging system; FIG. 4 is a flow chart showing an example of information processing of a client device; It is a figure which shows an example of a captured image. FIG. 10 is a diagram showing an example of a pattern other than pattern 2; 4 is a flow chart showing an example of information processing of a client device; FIG. 10 is a diagram showing an example of pattern 3; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 2; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 3; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 3; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 3; FIG. 10 is a diagram showing an example of pattern 3; It is a figure which shows an example of the image on which the distortion correction process was performed. FIG. 10 is a diagram showing an example of pattern 1; It is a figure which shows an example of the image on which the distortion correction process was performed.

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will be described below with reference to the drawings. A case where a camera is used as an imaging device will be described below.

<Embodiment 1>
FIG. 1 is a diagram showing an example of the hardware configuration of the camera 10. As shown in FIG. Camera 10 is a fisheye camera. The camera 10 includes an imaging unit 11, a CPU 12, a memory 13, an input unit 14, a display unit 15, and a communication unit 16 as a hardware configuration. The imaging unit 11 captures a subject image. Details of the imaging unit 11 are shown in FIG. 2, which will be described later. The CPU 12 controls the camera 10 as a whole. The memory 13 stores programs, images captured by the imaging unit 11, setting values, and the like. The input unit 14 inputs a user's selection operation and the like, and transfers the input to the CPU 12 . The display unit 15 displays a screen or the like under the control of the CPU 12 . A communication unit 16 connects the camera 10 to a network and controls communication with other devices. The CPU 12 executes processing based on the programs stored in the memory 13 to realize the software configuration of the camera 10 shown in FIG. 3 and the processing of the flow chart in FIG. Camera 10 is an example of an information processing device. The following embodiments can also be applied to a recording server that accumulates omnidirectional images acquired from the camera 10 via a network, or a terminal device that displays omnidirectional images.

FIG. 2 is a diagram showing an example of the imaging unit 11. As shown in FIG. The imaging unit 11 includes a lens 201 made up of several lens groups, and an imaging device 202 such as a CCD or CMOS. The imaging unit 11 also includes a correlated double sampling (CDS) circuit 203 that reduces noise. The imaging unit 11 also includes a gain control amplifier circuit (Automatic Gain Control: AGC) 204 that automatically performs gain control of the camera. The imaging unit 11 also includes an A/D converter 205 that converts an analog signal into a digital signal.

FIG. 3 is a diagram showing an example of the software configuration of the camera 10. As shown in FIG. The imaging control unit 101 controls the imaging unit 11 and passes signals from the imaging unit 11 to the image generation unit 102 . The image generator 102 generates a captured image based on the received signal. The determination unit 103 determines the imaging scene from the captured image. The pattern setting unit 104 sets a pattern for cutting out a partial image from the captured image and arranging the distortion-corrected image on the output image. The cutout position setting unit 105 sets the cutout position of the captured image according to the set pattern. A cropping unit 106 crops the image at the cropping position set by the cropping position setting unit 105 . A distortion correction processing unit 107 performs distortion correction corresponding to the image cut out by the cutout unit 106, and performs processing for arranging the cut out image as an output image based on the pattern set in the pattern setting unit 104. FIG. The output unit 108 outputs the image signal distortion-corrected by the distortion correction processing unit 107 .

The operation of each component described above will be described in detail.
First, the imaging device will be described in detail with reference to FIGS. 2 and 3. FIG. The imaging device 202 converts a subject image formed via a lens 201 as an imaging optical system into an electrical signal. The lens 201 may be a wide-angle lens or a fish-eye lens with a large distortion and a wide angle of view. In this embodiment, a case of a fish-eye camera provided with a fish-eye lens as an imaging device will be described. The CDS circuit 203 performs correlated double sampling processing and the like on the electrical signal output from the image sensor 202 . The AGC amplifier 204 performs amplification processing and the like on the electrical signal output from the CDS circuit 203 . The A/D converter 205 converts the analog signal amplified by the AGC amplifier 204 into a digital signal. The imaging control unit 101 passes the luminance signal and the color signal obtained from the imaging unit 11 to the image generation unit 102 . The image generator 102 generates an image signal such as an RGB image or a YUV image from the obtained luminance signal and color signal. The determination unit 103 determines the number of attention areas and the positions of the attention areas from the image generated by the image generation unit 102 . The pattern setting unit 104 sets a pattern based on the number and positions (or distribution) of the regions of interest in the image determined by the determination unit 103 . The cut position setting unit 105 sets a position at which the region of interest in the image determined by the determination unit 103 is cut, corresponding to the pattern set by the pattern setting unit 104 . At this time, a cut position is set so that one attention area is not divided. A cropping unit 106 crops a portion of the image generated by the image generating unit 102 using the cropping position set by the cropping position setting unit 105 . A distortion correction processing unit 107 performs distortion correction processing such as various geometric transformations on the image cut out by the cutout unit 106 . Then, based on the pattern set by the pattern setting unit 104, the distortion correction processing unit 107 executes processing for arranging the distortion-corrected image as an output image.

Here, an image cut out by the cutout unit 106 and an image subjected to distortion correction by the distortion correction processing unit 107 will be described with reference to FIGS. 4 to 7. FIG. An omnidirectional image 30 in FIG. 4 is an example of an image obtained by capturing an image in a direction perpendicular to the floor with a fisheye camera having a fisheye lens installed on the indoor ceiling. The clipping unit 106 excludes a region 300 delimited by dashed lines in the omnidirectional image 30 of FIG. 4 and cuts regions 301 and 302 delimited by dashed lines, respectively. FIG. 5 shows an example of an output image in which the distortion correction processing unit 107 performs distortion correction processing, and the region 302 is vertically inverted and connected. FIG. 4 shows an example in which an area 300 is regarded as an invalid area and is not output as an output image, and an area 301 and an area 302 including the attention area are regarded as effective areas and output as an output image. Furthermore, as another example, as shown in FIG. 6, an example is conceivable in which the clipping unit 106 partially clips regions 400 to 402, which are regions of interest, and the distortion correction processing unit 107 performs distortion correction processing. In areas 400 to 402 in FIG. 7, images output by performing distortion correction processing on the images of the cutout areas 400 to 402 shown in FIG. 6 are arranged. In the example of FIG. 6, areas 400 to 402 in FIG. 6 are regarded as valid areas and used as the output image, and other areas are regarded as invalid areas and not used as the output image. Note that the output unit 108 may synthesize three partial images and output them as one image, or may output these three partial images as a plurality of streams. In the case of four or more, for example, the clipped images may be combined and output after being scaled according to the size of the image output by the output unit 108, or may be output as multiple streams. The output unit 108 outputs the image signal that has undergone distortion correction processing by the distortion correction processing unit 107 . The output destination of the image signal may be a display built into the camera 10 or an external terminal device connected to the network via the communication unit 16 .

An example in which the camera 10 sets a pattern and a cutout position for outputting an image according to the condition of the attention area of the captured image will be described with reference to the flowchart of FIG. 8 . In S<b>101 , the determination unit 103 acquires an image generated by the image generation unit 102 . This image is an omnidirectional image before distortion correction processing. Next, in S102, the determination unit 103 extracts a region of interest from the obtained omnidirectional image before distortion correction processing. Next, in S103, the determination unit 103 determines whether or not there is one or more regions of interest. The determination unit 103 may calculate the distribution of faces and persons in the image, and may determine an area in the image in which the number of faces and persons is greater than a set number as an attention area. A region higher than the value may be determined as a region of interest. Further, the determination unit 103 may calculate the retention frequency and flow line density of a face or a person from a single image, or may calculate the retention frequency and flow line density of a face or person from a plurality of images. may Note that when detecting a face or a person using pattern matching, it is necessary to detect the face or person from an image that has undergone distortion correction. Further, the determination unit 103 may simplify the analysis and count the number of foreground regions having a predetermined size or larger. In this embodiment, the determination unit 103 determines a region having a predetermined size and including a predetermined number (for example, three) or more of people as a region of interest in the peripheral region of the omnidirectional image before distortion correction processing. I judge. On the other hand, the determination unit 103 determines, as a region of interest, a region in which one or more persons exist within the region having the set size with respect to the central region in the omnidirectional image before the distortion correction processing. This is because, in the image before distortion correction processing, the person existing in the central portion is captured larger than that in the peripheral portion. The set size of the above-described area may be determined in advance as a proportion of the entire image and stored in the memory 13, or may be stored in the memory 13 or the like in response to a user operation via the input unit 14 or the like. may be stored. If the attention area does not exist on the image (No in S103), the determination unit 103 transitions to S104. In S104, the pattern setting unit 104 sets a pattern preset by the user. Also, the cutting position setting unit 105 sets the cutting position corresponding to the pattern. Here, the pattern preset by the user may be any one of pattern 1, pattern 2, and pattern 3, or may be an omnidirectional image before cropping. In this embodiment, it is assumed that an omnidirectional image before cropping in which the entire image can be overlooked is used as a pattern preset by the user, and this omnidirectional image is output. In S105, the determination unit 103 determines whether or not there are two or more regions of interest. If two or more attention areas do not exist (No in S105), the determination unit 103 transitions to S106, and if two or more attention areas exist (Yes in S105), transitions to S107.

In S<b>106 , the pattern setting unit 104 sets pattern 1 for outputting one image without outputting a divided image because it corresponds to one attention area. Also, the cutout position setting unit 105 sets a cutout position that includes one attention area so as to correspond to the set pattern 1 . In S107, the determination unit 103 determines whether or not there are three or more regions of interest. If there are not three or more attention areas (No in S107), the determination unit 103 transitions to S108, and if there are three or more attention areas (Yes in S107), the determination unit 103 transitions to S109. In S108, the pattern setting unit 104 sets pattern 2 for generating and outputting a two-screen divided image (double panorama) so as to correspond to two attention areas. In addition, the cutout position setting unit 105 sets cutout positions so that each of the detected attention areas is not divided so as to correspond to the set pattern 2 . In S109, the pattern setting unit 104 sets pattern 3 for generating and outputting four-screen split images so as to correspond to three or more attention areas. Also, the cutout position setting unit 105 cuts out the attention area so as to correspond to the set pattern.

An example in which the camera 10 cuts out an image based on the pattern and cutout position set by the camera 10, performs distortion correction, and outputs the image will be described with reference to the flowchart of FIG. In S<b>150 , the cutting unit 106 cuts out a region of interest from the captured image based on the pattern set by the pattern setting unit 104 and the position set by the cutting position setting unit 105 . In the example of the pattern 1 and the cutting position set in S106, the cutting unit 106 cuts out the region of interest 501 located at the upper left surrounded by one dashed line from the whole image of FIG. In addition, in the example of the pattern 2 and the cutting position set in S108, the cutting unit 106 removes the middle invalid region surrounded by two dashed lines from the entire image of FIG. Crop a partial image from 602 . That is, the cutout position is set so that each of the two cutout images includes one region of interest. In addition, the cutting unit 106 cuts three regions of interest 701, 702, and 703 surrounded by broken lines from the entire image in FIG. 14 in the example of the pattern 3 and the cutting position set in S109. In S151, the distortion correction processing unit 107 performs distortion correction processing on the image cut out in S150. For example, when the pattern is pattern 1 or pattern 3, the distortion correction processing unit 107 executes distortion correction processing according to the pattern. Further, for example, when the set pattern is pattern 2, the distortion correction processing unit 107 executes distortion correction processing according to the pattern and the cutout position. Here, the distortion correction processing is, for example, processing such as image enlargement/reduction and geometric transformation. In S152, the output unit 108 outputs the image subjected to the distortion correction processing in S151. FIG. 11 is a diagram showing an example in which the image of the attention area 501 in FIG. 10 is cropped according to the settings in S106, subjected to distortion correction processing, and output to the client device 20 shown in FIG. 17, which will be described later. FIG. 13 is a diagram showing an example in which the regions 601 and 602 in FIG. 12 are cut out according to the settings in S106, subjected to distortion correction processing, and output. As shown in FIG. 13, in pattern 2, a two-screen split image called double panorama is output. FIG. 15 is a diagram showing an example in which the attention areas 701, 702, and 703 in FIG. 14 are cut out according to the settings in S109, subjected to distortion correction processing, and output. That is, in pattern 3, four-screen split images are output. Note that when pattern 3 is selected and there are only three attention areas, an empty area is generated. In this case, the output unit 108 rearranges the cutout images 701, 702, and 703 as shown in FIG. 16, and reduces the omnidirectional image, which is the captured image, to the empty area as shown in 801 in FIG. can be output as This omnidirectional image is effective when there are five or more attention areas. Note that in the flowchart shown in FIG. 8, pattern 3 is selected when there are three or more attention areas. When there are five or more attention areas, if attention areas are assigned to divided images one by one, there will be attention areas that cannot be output. In this embodiment, when there are five or more attention areas, adjacent attention areas may be grouped to create four groups. In addition, when there are five or more attention areas, or when the attention areas overlap and the optimum cutout position cannot be set, the entire image can be viewed from above without outputting partial images. Images may be output.

Each pattern in this embodiment is an example of cutting. Other patterns with different numbers of cuts and cut positions may be used. Moreover, the change timing of each pattern may be immediately reflected in the determination result of the determination unit 103, may be set at a predetermined constant interval, or may be arbitrarily changed by the user. A distortion correction processing unit 107 performs distortion correction processing on the cropped image, and an output unit 108 outputs the image.

As described above, according to the processing of the present embodiment, it is possible to prevent the region of interest (object of interest) from being cut off due to image clipping. In addition, it is possible to output an image that has undergone appropriate cropping processing and distortion correction processing in accordance with the imaging scene or the like without user setting. Therefore, since the pattern of the image and the distortion correction process are appropriately changed according to the imaging scene or the like, it is possible to provide an image suitable for monitoring.

<Embodiment 2>
FIG. 17 is a diagram illustrating an example of a hardware configuration of devices that constitute an imaging system. In the imaging system, a camera 10 and a client device 20 are communicably connected via a network. The network may be wired or wireless. The hardware configuration of the camera 10 is the same as that of the first embodiment. The CPU 12 executes processing based on the programs stored in the memory 13, thereby realizing the software configuration of the camera 10 shown in FIG. The client device 20 includes a CPU 21, a memory 22, an input section 23, a display section 24, and a communication section 25 as a hardware configuration. The CPU 21 controls the client device 20 as a whole. The memory 22 stores programs, images transmitted from the camera 10, setting values, and the like. The input unit 23 inputs a user's selection operation and the like, and transfers the input to the CPU 21 . The display unit 24 displays a screen or the like under the control of the CPU 21 . The communication unit 25 connects the client device 20 to a network and controls communication with other devices (for example, the camera 10). When the CPU 21 executes processing based on the program stored in the memory 22, the software configuration of the client device 20 shown in FIG. 18 and the processing of the flow chart in FIG. The client device 20 is an example of an information processing device.

FIG. 18 is a diagram showing an example of the software configuration of the devices that make up the imaging system. The imaging unit 11 and the image generating unit 102 of the camera 10 have the same configurations as those of the first embodiment, so description thereof will be omitted. The imaging direction detection unit 121 detects the imaging direction of the imaging unit 11 . For example, the imaging direction detection unit 121 may acquire the imaging angle from the output of the gyro sensor. The determination unit 103, the pattern setting unit 104, the cutout position setting unit 105, the cutout unit 106, the distortion correction processing unit 107, and the output unit 108 of the client device 20 are the same as those of the camera 10 of the first embodiment, so description thereof will be omitted. omitted. The communication processing unit 122 communicates with the camera 10 to receive the image generated by the image generation unit 102 and the imaging direction detected by the imaging direction detection unit 121 .

Next, using the flowchart shown in FIG. 19, an example of setting and outputting an image pattern and a cropping position according to the condition of the attention area of the captured image by the imaging system configured by the camera 10 and the client device 20 will be described. explain. Description of S101 to S109 in the flowchart of FIG. 19 is omitted because the processing contents and processing flow are the same as those in the first embodiment. 10 to 15 are the same as the processing contents of the first embodiment, so description thereof will be omitted.
In S<b>201 , the determination unit 103 determines whether the imaging direction of the imaging unit 11 is horizontal based on the imaging direction of the imaging unit 11 detected by the imaging direction detection unit 121 and received via the communication processing unit 122 . At this time, the imaging direction is the horizontal direction, for example, when the camera 10 is installed on a wall surface, and in other cases, such as when it is installed on the ceiling and is imaging in the vertical direction. If the determination unit 103 determines that the imaging direction is horizontal, the process transitions to S106. If the determination unit 103 determines that the imaging direction is not horizontal, the process transitions to S108. In pattern 2, the peripheral images excluding the central portion of the entire image are cut and distorted. Therefore, when the imaging direction is the horizontal direction, the pattern setting unit 104 sets pattern 1 in S106. In addition, the cutout position setting unit 105 sets a cutout position for cutting out one partial image including two attention areas so as to correspond to the set pattern 1 . At this time, since the attention area is considered to appear in the central portion of the omnidirectional image excluding the upper and lower regions, the cutout position is set so as to cut out the central portion of the omnidirectional image. On the other hand, if the imaging unit 11 is not in the horizontal direction, that is, in the vertical direction, the pattern setting unit 104 sets pattern 2 in S108. In addition, the cutout position setting unit 105 sets the cutout position so that each of the two cutout partial images includes one region of interest so as to correspond to the set pattern 2 . In S202, the determination unit 103 determines whether or not an attention area such as a specific moving object has appeared in the central portion of the entire image as shown in FIG. When the attention area such as the specific moving body appears in the central portion of the entire image (Yes in S202), the determination unit 103 transitions to S203, and determines that the attention area such as the specific moving object does not appear in the central portion of the entire image. If so (No in S202), the process of the flowchart of FIG. 19 is terminated. In S203, the pattern setting unit 104 temporarily sets a pattern other than pattern 2, for example, pattern 3 as shown in FIG. In addition, the cropping position setting unit 105 sets the cropping positions of the three partial images so that each of the three partial images includes the attention area according to the set pattern 3 . Here, the determination unit 103 may recognize a specific subject as a specific moving object by face recognition or the like, or may identify a moving object when the size or speed of the moving object exceeds a predetermined threshold value or a threshold value set by the user. may be recognized as a moving object. Also, the central portion of the image is, for example, within a range set from the center of the image. Pattern 2 is an example of a first pattern. The processing of S203 is an example of processing for setting a pattern other than the first pattern.

As described above, according to the processing of this embodiment, in addition to the effects of the first embodiment, it is possible to provide an image suitable for monitoring a specific moving object.

<Embodiment 3>
Embodiment 3 will describe an example of a case different from Embodiments 1 and 2 when a pattern and a cutout position corresponding to the pattern are set according to the conditions of the attention area of the captured image. The hardware configuration and software configuration of the camera 10 and the client device 20 are the same as those in the above-described embodiment, and thus description thereof will be omitted. The CPU 21 executes the processing based on the program stored in the memory 22 to implement the processing of the flowchart of FIG. 22, which will be described later.
Next, using the flowchart shown in FIG. 22, an example in which the imaging system configured by the camera 10 and the client device 20 sets the image pattern and the clipping position according to the conditions of the attention area of the captured image, and outputs them. explain. S106 to S109 and S201 of the flowchart are the same processing contents and processing flow as those of the first and second embodiments, respectively. Also, the processing of S101 to S105, S202, and S203 described in the flowcharts of the first and second embodiments may be added to FIG.
The patterns and the like in FIGS. 23 to 28 are the same as those in the first embodiment, so description thereof will be omitted.

In S301, the determination unit 103 determines whether or not there are five or more regions of interest. If five or more attention areas do not exist (No in S301), the determination unit 103 transitions to S109, and if five or more attention areas exist (Yes in S301), transitions to S302. In S109, when there are four regions of interest less than five regions as shown in FIG. 23, the pattern setting unit 104 selects pattern 3 as shown in FIG. do. Also, although the number of regions of interest is five, any number may be used as long as it exceeds the number of divisions that can be divided by the pattern setting unit 104 . In S302, the determination unit 103 determines whether a priority area is set in the captured image. If the priority area is not set in the captured image (No in S302), the determination unit 103 proceeds to S303, and if the priority area is set in the captured image (Yes in S302), the process proceeds to S306. Transition. FIG. 25 shows an example in which there are five regions of interest, and the determination unit 103 determines the positional relationship of the five regions. Since pattern 2 can output all five regions of interest, the pattern setting unit 104 selects pattern 2 as shown in FIG. 26 in S303. At this time, the cutout position setting unit 105 sets the cutout position at a position where each attention area is not divided. As a result, the image cropping unit 106 crops the regions 1101 and 1102 at positions where the regions of interest are not divided. Further, the distortion correction processing unit 107 performs distortion correction processing. In S<b>304 , the determination unit 103 determines whether or not the subject, which is the attention area, has entered the central area 1103 as shown in FIG. 27 . If the subject has entered the central area (Yes in S304), the determining unit 103 proceeds to S305. end the processing of In S305, the pattern setting unit 104 may temporarily select an omnidirectional image as an area 1104 that can include all areas 1101 to 1103 in FIG. 27, as shown in FIG. Also, when the subject disappears from the area 1103, the pattern setting unit 104 resets the pattern 2 as shown in FIG.

In S306, when the detection area is set in the captured image, the determination unit 103 determines whether or not the detection target exists within the set detection area. If the detection target does not exist within the detection area preset in the captured image (No in S306), the determination unit 103 transitions to S307, and detects within the detection area preset in the captured image. If the target exists (Yes in S306), the process proceeds to S308.
FIG. 29 is an example showing a case where the priority area is set and the detection area is not set. A region 1205 in FIG. 29 is set as a priority region. A priority area is an area in which a partial image corresponding to the priority area is cut out and output regardless of whether or not the area is detected as an attention area. However, the priority area can also be set to any area in the image. Also, a plurality of priority areas may be set. FIG. 31 also shows an example of a detection area 1206 set in the image. The detection area here is an area for detecting a specific event such as a person entering or disappearing from the area. FIG. 32 shows that an intrusion of a person has been detected within the detection area 1206 .
In S308, the pattern setting unit 104 selects the priority area 1205 (1202) and the detection area 1206 (1207) shown in FIG. select. Then, the pattern setting unit 104 sets the pattern 3 so that the images are arranged as shown in FIG. At this time, among the attention areas, areas 1201 and 1203 with a large number of people are selected as areas with a higher degree of attention. Then, the cutout position setting unit 105 sets cutout positions at which the partial images of the priority region 1205 , the detection region 1206 , and the regions 1201 and 1203 are cut out. As a result, the image cropping unit 106 crops partial images from areas 1201 , 1203 , 1205 and 1206 . In this processing, when a priority area is set in the omnidirectional image, the image cropping unit 106 preferentially selects the partial image corresponding to the priority area regardless of whether the priority area is detected as the attention area. This is an example of a process of cutting out to . Further, this processing is performed when the detection area is set in the omnidirectional image, and when the image cropping unit 106 detects the detection area as an attention area with a lower attention degree than other attention areas. is also an example of processing for preferentially cutting out a partial image corresponding to the detection area. A distortion correction processing unit 107 performs distortion correction processing on each cut partial image.
In S307, if no detection area is set, a priority area 1205 shown in FIG. 29 is selected, and areas 1201, 1203, and 1204 with a higher degree of attention are selected from the attention areas extracted in step S102. do. Then, the pattern setting unit 104 sets the pattern 3 so as to have the arrangement shown in FIG. In FIG. 29, since no detection area is set, the attention area in FIG. 29 existing at the same position as the area 1207 in FIG. 32 is not selected, and the attention area 1204 is selected instead. The reason is that the number of people in the attention area 1204 is larger than the number of people in the attention area of FIG. 29 existing at the same position as the area 1207 of FIG. Note that there is also an attention area having the same number of people as the attention area 1204 on the left side of FIG. will be selected.
Also, the cutout position setting unit 105 sets the cutout positions so that the partial images corresponding to the selected areas 1201, 1202 (1205), 1203, and 1204 are cut out.

In S309, the determination unit 103 detects two attention areas and determines whether or not the two detected attention areas are close to each other. The determination unit 103 determines that two regions of interest are close to each other when two regions of interest exist within one semicircle (excluding the central region) of the circular captured image of the fisheye camera before distortion correction. . If the determining unit 103 determines that the two attention areas are close to each other as shown in FIG. 34, the process proceeds to S106. That is, pattern 1 is selected, and a semicircle excluding the central region is set as the cut position. The image after distortion correction becomes a single panorama image as shown in FIG. On the other hand, if the determining unit 103 determines that the two attention areas are not close to each other as shown in FIG. 12, the process proceeds to S108. As a result, a double panorama image as shown in FIG. 13 is displayed.

As described above, according to the processing of this embodiment, in addition to the effects of the first and second embodiments, it is possible to provide an image suitable for monitoring in consideration of a specific area, an object, and a moving body.
It should be noted that the selection of patterns described in Embodiments 1 to 3 can be combined as appropriate.
<Other embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or storage medium. It can also be realized by a process in which one or more processors in the computer of the system or device reads and executes the program. It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

Although one example of the embodiment of the present invention has been described in detail above, the present invention is not limited to such a specific embodiment.
For example, in the above-described embodiments, the camera described as the imaging device can be applied to a digital still camera or a digital video camera.
Also, part or all of the software configuration described above may be implemented in the device as a hardware configuration.
Also, as a hardware configuration, a GPU (Graphics Processing Unit) may be used in place of the CPU.

As described above, according to each of the above-described embodiments, it is possible to output an image that has undergone appropriate cropping processing and distortion correction processing in real time according to the imaging scene or the like without user's setting.

10 camera 11 imaging unit 12 CPU

Claims (7)

setting means for setting a pattern for outputting a partial image obtained from the omnidirectional image based on the number and positions of attention areas obtained from the omnidirectional image, which is a captured image captured using a fisheye lens;
a cutting means for cutting a partial image including the attention area from the omnidirectional image based on the pattern set by the setting means;
correction means for performing distortion correction processing on the partial image cut out by the cutout means;
output means for outputting a partial image that has undergone distortion correction processing by the correction means according to the pattern set by the setting means;
has
The setting means further sets the pattern based on the imaging direction of the imaging unit having the fisheye lens, cuts out one partial image when the imaging direction is the horizontal direction, and the imaging direction is the vertical direction. , the information processing apparatus sets a pattern for cutting out two partial images . When there are two attention areas obtained from the omnidirectional image, the setting means cuts out one partial image including the two attention areas if the positions of the two attention areas are close to each other. 2. The information processing apparatus according to claim 1, wherein if not, a pattern is set for cutting out two partial images each containing one attention area. 3. The information processing device according to claim 1 , wherein said information processing device is an imaging device. The information processing apparatus according to any one of claims 1 to 3 , wherein the information processing apparatus is a client apparatus capable of communicating with an imaging apparatus. setting means for setting a pattern for outputting a partial image obtained from the omnidirectional image based on the number and positions of attention areas obtained from the omnidirectional image, which is a captured image captured using a fisheye lens;
a cutting means for cutting an image of a portion including the attention area from the omnidirectional image based on the pattern set by the setting means;
correction means for performing distortion correction processing on the partial image cut out by the cutout means;
output means for outputting an image subjected to distortion correction processing by the correction means according to the pattern set by the setting means;
has
The setting means further sets the pattern based on the imaging direction of the imaging unit having the fisheye lens, cuts out one partial image when the imaging direction is the horizontal direction, and the imaging direction is the vertical direction. , a system that sets a pattern for cropping two partial images . An information processing method executed by an information processing device,
a setting step of setting a pattern for outputting a partial image obtained from the omnidirectional image based on the number and positions of attention areas obtained from the omnidirectional image, which is a captured image captured using a fisheye lens;
a cutting step of cutting a partial image including the attention area from the omnidirectional image based on the pattern set in the setting step;
a correction step of performing distortion correction processing on the partial image cut out in the cutout step;
an output step of outputting a partial image that has undergone distortion correction processing in the correction step according to the pattern set in the setting step;
including
The setting step further includes setting the pattern based on the imaging direction of the imaging unit having the fisheye lens, cutting out one partial image when the imaging direction is horizontal, and setting the imaging direction to the vertical direction. , an information processing method for setting a pattern for cutting out two partial images . A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 4 .

Images