JP7815026B2 - Filming systems, video server systems - Google Patents

Description

The present invention relates to a photography system that captures images and a video server system that receives the resulting video signals and performs various processes.

A wireless relay device for television broadcasting (FPU: Field Pickup Unit) relays digitized signals of video captured outside the broadcast station when they are transmitted to the broadcast station. By enhancing the functionality of the FPU device, stable, high-quality broadcasting can be achieved. For example, Patent Document 1 describes an FPU device that automatically optimizes the transmission rate of digitized signals based on the video signal. This allows the video signal to be transmitted to the broadcast station in good transmission conditions.

Japanese Patent Application Laid-Open No. 2004-104579

The technology described in Patent Document 1 optimizes the transmission conditions of a video signal by examining the video signal itself to be transmitted. In this way, there was a demand for technology that not only optimizes transmission conditions based on the obtained video signal, but also improves the functionality and performance of the imaging system.

The present invention was developed in light of these circumstances and aims to solve the above problems.

The present invention is a photography system that uses an imaging device that captures images and outputs a video signal, wherein the imaging device has an imaging information output unit that outputs imaging information that includes positional information of the imaging device at the time of imaging and indicates the state of the imaging device at the time of imaging, and is equipped with: a communication unit that obtains the video signal and the imaging information from the imaging device; and an analysis unit that recognizes the subject in the image generated by the video signal, recognizes map information about the area around the imaging device based on the positional information, including information about structures around the imaging device, and recognizes the correspondence between the subject and the structure by comparing the image with the map information, wherein when part of the information other than the positional information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates the imaging range for each parameter in the map information captured by the imaging device, and recognizes the correspondence between the subject and the structure in the imaging range, and determines that the value of the parameter when the subject and the structure best match corresponds to the value of the missing information, and excludes structures that are determined not to be recognized in the imaging range based on information other than the missing information when recognizing the correspondence .
The image capturing apparatus may further include a display unit that displays the image while indicating the correspondence between the subject and the structure.
In addition, the analysis unit may recognize in advance the object to be imaged by the imaging device, acquire the map information of the surroundings of the object, and by comparing the map information with the image based on the video signal obtained from the imaging device, issue an instruction to the imaging device to control the imaging device so that the object is imaged.
The imaging information may also include any of the altitude of the imaging device, the direction of the center of the field of view when capturing an image, the tilt angle around the optical axis, the field of view range, and the distance to the imaged object.
The analysis unit may also register identification information of the structure recognized as the subject.
The system may also include a video server system having the imaging device, the communication unit, and the analysis unit, and a transmission device that obtains the video signal and the imaging information from the imaging device and transmits them to the video server system by wireless communication via a directional antenna, wherein the imaging device is fixed to the transmission device and an adjustment display unit is provided that displays the orientation of the antenna in the video that includes the subject whose correspondence with the structure has been identified.
The present invention provides
A video server system that obtains and processes video signals obtained by an imaging device, comprising: a communication unit that receives from the imaging device, together with the video signal, imaging information that includes position information of the imaging device at the time of imaging and indicates the state of the imaging device at the time of imaging; and an analysis unit that recognizes subjects in the video from the video signal, recognizes map information around the imaging device from the position information including information about structures around the imaging device, and recognizes the correspondence between the subjects and the structures by comparing the video with the map information ; when part of the information other than the position information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates the imaging range for each of the parameters in the map information captured by the imaging device, recognizes the correspondence between the subjects and the structures in the imaging range, and determines that the value of the parameter when the subject and the structure best match corresponds to the value of the missing information; and excludes structures that are determined not to be recognized in the imaging range based on information other than the missing information when recognizing the correspondence .

This invention makes it possible to improve the functionality and performance of the imaging system based on the obtained video signal.

FIG. 1 is a diagram showing a configuration of an imaging system according to an embodiment. FIG. 1 is a diagram showing a configuration of an imaging device used in an imaging system according to an embodiment. FIG. 1 is a diagram showing the configuration of a video server system used in a photography system according to an embodiment. 1 is an example of an image obtained by the imaging system according to the embodiment. 10 is an example of map information used in the imaging system according to the embodiment. 10 is an example in which a virtual image obtained by appropriately setting parameters is displayed together with an actual image in the imaging system according to the embodiment. FIG. 10 is a diagram showing a case where the field of view range in the horizontal direction is determined in map information. FIG. 10 is a diagram showing a case where the field of view range in the vertical direction is determined in map information. FIG. 10 is a diagram showing ranges corresponding to depth of field in map information. 10 shows an example of an image in which the focus conditions for each subject in the image are different. This is an example in which the distance to each subject in the video is recognized. This is an example in which the distance to each structure in the map information is recognized. 10A shows an example of an image with optical distortion, and FIG. 10B shows an example of an image after correction. 10A and 10B are diagrams illustrating operations when issuing instructions to bring a desired object into the field of view. 10A shows the configuration of the imaging device when performing antenna adjustment, and FIG. 10B shows the image display.

Next, a photography system that is an embodiment of the present invention will be described in detail with reference to the drawings. As shown in Figure 1, this photography system 100 comprises an imaging device 10 that captures images of a subject to obtain video data (generates a video signal), an FPU unit (transmission device) 30 that digitizes the video signal and transmits (relays) it to a broadcast station, and a video server system 40 that receives the video signal from the broadcast station, performs various processes on the video, and stores it. The imaging device 10 and FPU unit 30 are mounted on, for example, a photography vehicle or helicopter, and the video server system 40 is connected to the FPU unit 30 via wireless communication and is fixed at a fixed location on the ground (for example, a broadcast station).

In this photography system 100, the imaging device 10 recognizes its own position information and imaging conditions (posture, zoom magnification, etc.), and the imaged subject is recognized using AI technology, etc., thereby recognizing the subject's position information. This information is transmitted to the video server system 40 along with the imaging data. The video server system 40 can then use this information to perform various types of processing.

Figure 2 is a diagram showing the configuration of this imaging device 10. In this imaging device 10, an image is captured by the imaging element 12 via the optical system 11, generating an analog video signal. This video signal undergoes various pre-transmission processes in the video signal processing unit 13, and is then output externally as a digital video signal conforming to HD-SDI standards or the like from the video signal output unit 14, which is received by the FPU unit 30. The imaging device 10 also includes a control unit (imaging device control unit) 15 that controls the entire imaging device. The control unit 15 controls the optical system 11 and the imaging element 12, and controls the field of view (zoom magnification, etc.), focal length, aperture, gain, etc. during imaging.

The imaging device 10 is also provided with various components to recognize the state of the imaging device 10 itself during imaging. First, as shown in FIG. 2, the imaging device 10 is provided with a location information recognition unit 16 that recognizes the location (location information) of the imaging device 10 by receiving GPS signals, and an altitude sensor 17 that recognizes the altitude (elevation) of the imaging device 10. These components enable the control unit 15 to recognize the location of the imaging device 10 in three-dimensional space during imaging.

Also shown in Figure 2 is an orientation sensor 18 that recognizes the orientation (azimuth angle) of the optical axis (center of field of view) of the imaging device 10, and an inclination sensor 19 that recognizes the elevation angle of the optical axis of the imaging device 10 from the horizontal direction and the inclination angle around the optical axis. These sensors allow the control unit 15 to recognize the direction in three-dimensional space of the center of field of view of the imaging device 10 and the inclination angle around the optical axis.

The control unit 15 can also recognize the field of view range around the center of the field of view in the horizontal and vertical directions from the control information of the optical system 11 and the image sensor 12. Note that the field of view range recognized here in the horizontal and vertical directions is the field of view range when the tilt angle around the optical axis recognized by the tilt sensor 19 is zero, as described above; if this tilt angle is not zero, the field of view range will have a tilted shape according to the tilt angle.

Also shown in Figure 2 is an illuminance sensor 20 that recognizes the illuminance of the environment around the imaging device 10, and a time recognition unit 21 that recognizes the current date and time.

With the above configuration, when the imaging device 10 captures an image, the control unit 15 can recognize the position information (horizontal position and height) of the imaging device 10 in three-dimensional space and field of view information (direction of the field of view center and extent of the field of view in three-dimensional space). The control unit 15 can also recognize the time when the image was captured and the illuminance, and can output all of this recognized information together with the video data from the imaging information output unit 22 to the outside as imaging information attached to the video data.

In the above configuration, the altitude sensor 17, illuminance sensor 20, and time recognition unit 21 may not be provided within the imaging device, but may be provided on the helicopter or other device that carries the imaging device. Even in this case, the information obtained from these sensors can be included in the imaging information as described above.

However, as mentioned above, in the imaging device 10, each item in the imaging information is recognized by a different component, and it may be possible to obtain information on only some of these items with high accuracy. In the examples described below, we will explain the case where information on only some of these items is obtained.

In Figure 1, the imaging device 10 outputs video signals and imaging information as described above, and the FPU device 30 receives these signals via a wired connection from the imaging device 10, which is located nearby, and transmits them via wireless communication to the video server system 40, which is located in a remote location. At this time, the FPU device 30 modulates these signals according to an appropriately selected transmission method, and the video server system 40 demodulates them. However, since the configurations related to this modulation and demodulation are well known and unrelated to the present invention, they will not be described below.

Figure 3 shows the configuration of the video server system 40. This system is equipped with a wireless communication unit (communication unit) 41 that receives the video signal and imaging information from the FPU device 30 via wireless communication. Instructions for various controls on the FPU device 30 are sent from the video server system 40 to the FPU device 30 via wireless communication or other communication means by the wireless communication unit 41. A control unit (analysis unit) 42 is also provided to control the entire video server system 40.

When the control unit 42 recognizes the location information of the imaging device 10 from the acquired imaging information, it can obtain map information of its surroundings (the area that may be captured by the imaging device 10) via the network using the network connection unit 43. The acquired video signal, imaging information, and map information are stored in the storage unit 44, which is a large-capacity hard disk or non-volatile memory. Note that if the imaging device 10 (FPU device 30) captures images while moving, map information can be acquired and stored, for example, for each area in which the imaging device 10 is located. Alternatively, if the target to be imaged is predetermined, map information including this target may be stored in the storage unit 44 in advance, rather than obtaining map information via the network connection unit 43 after recognizing the location information.

The control unit 42 can also display an image based on the video signal on the display unit 45. The control unit 42 analyzes this image based on map information and imaging information, and can, for example, recognize structures or the like (buildings, specific terrain, bridges, etc.) as subjects in the image, and can specifically identify these structures or the like by matching them with map information using the imaging information. This operation is explained in detail below. This operation can be performed periodically (for example, every second) when performed in real time when acquiring a video signal (video), but can also be performed on a video signal stored in the memory unit 44, as will be described later.

Figure 4 shows a schematic example of an image obtained from the imaging device 10 (video signal) in this case. The subject here is a large structure such as a building, which can be recognized using well-known pattern recognition techniques. Three subjects a1 to a3 are recognized here. Note that the grid lines in Figure 4 are virtual and correspond to the coordinate display in Figure 5, which will be described later. The control unit 42 can specifically identify subjects a1 to a3 in this image by comparing this image with the map information described above. As mentioned above, not all items of information may be obtained from the imaging information, so below we will explain analysis based on the information for each item.

First, we will explain the case where the position information of the imaging device 10, particularly the horizontal position information, has been determined from the imaging information. In this case, the control unit 42 obtains map information about the area around this position. Figure 5 shows the map information obtained in this case and the position P of the imaging device 10 within it. Here, the horizontal axis is divided into three sections, A to C, and the vertical axis is divided into nine sections, 1 to 9, with the imaging device 10 located at the center of "B8." The control unit 42 can store this map information in the memory unit 44.

Here, buildings and other structures (structures) are candidates for the aforementioned subjects a1 to a3 that can be recognized in the image captured by the imaging device 10 (Figure 4) using well-known pattern recognition technology. Here, six structures G, H, I, X, Y, and Z are located in the locations shown.

In this case, the image captured by the map information terrain, assuming the imaging device 10 is in this position, is simulated using various conditions at the time of imaging as parameters, and the simulated image is compared with the actual image (Figure 4). The parameters that produce the closest result can be estimated to be the actual ones. These parameters include the altitude of the imaging device 10, the direction of the field of view center, the field of view range, and the tilt angle around the optical axis. In this case, the correspondence between the subjects a1 to a3 recognized in Figure 4 and the structures G, H, I, X, Y, and Z in Figure 5 can be recognized.

Figures 6(a) and (b) show examples of the results of comparing each structure to be recognized in a virtual image when these parameters are appropriately set with the actual image (Figure 4). This display is performed, for example, on the display unit 45. Here, the actual image is depicted with solid lines, and the virtual image is depicted with dotted and dashed lines. Here, structures G, I, and H are considered as candidates for subjects a1 to a3. In Figure 6(a), the angle in the vertical direction at the center of the field of view differs from that of the actual image, and in Figure 6(b), the tilt angle around the optical axis differs from that of the actual image. The virtual image to be compared with the actual image is recognized from map information, and the type of information that allows for easy comparison, such as a topographical map or aerial photograph, can be used as appropriate.

Here, by excluding as candidates structures that are clearly outside the field of view based on the position information of the imaging device 10, the number of structures that are candidates for the subject can be reduced. For example, based on the horizontal position of the imaging device 10 (Figure 5), structure X is in the shadow of structure I, so structure X can be excluded from consideration.

The control unit 42 performs this comparison and can estimate that the parameters (center of field of view, tilt angle around the optical axis) that best match the results in Figure 4 are actual, and can recognize, for example, that subjects a1, a2, and a3 in Figure 4 correspond to structures G, I, and H in Figure 5, respectively.

Next, we will discuss the case where the horizontal field of view range can be recognized in addition to the horizontal position of the imaging device 10. Figure 7 shows an example in which the horizontal field of view range R1 is added to the state shown in Figure 5. In this case, the range R1 is specified as the field of view range. In this case, altitude (vertical position information) and vertical field of view information (direction of the field of view center, field of view range) are used as parameters, and an image of the terrain of this map information is simulated as in Figure 6. The simulated image is compared with the actual image (Figure 4). The altitude, etc., resulting from the closest result is assumed to be the actual one, and the correspondence between the subjects recognized in Figure 4 and structures G, H, I, X, Y, and Z in Figure 5 can be recognized. In this case, structure X, which is in the shadow of structure I as described above, and structure Z, which is clearly outside the field of view range R1 in Figure 7, can be excluded as candidates for the subjects recognized in Figure 4. Only structures G, H, I, and Y that were not excluded can be selected as candidates for subjects a1 to a3, and the correspondence between the subjects and structures can be examined as in Figure 6.

Next, we will explain the case where the altitude (vertical position information) and vertical field of view information of the imaging device 10 can be recognized in addition to the horizontal position information and horizontal field of view information of the imaging device 10. Figure 8 is a side view of the vertical situation in this case, seen from the right side in Figure 7. The imaging device 10 is located at a predetermined height h from the ground at "B8" ("8" in Figure 8), and the control unit 42 can recognize this height from the imaging information. The vertical field of view range of the imaging device 10 is R2 in Figure 8. The height (total height) from the ground of each structure in Figure 8 is assumed to be as shown, and this can be recognized from map information.

In this case, as in the previous case, structures X and Z in Figure 8 can be excluded as candidates. Furthermore, structure Y also falls outside the vertical field of view range R2 in Figure 8, so it can be excluded as a candidate close to the imaging device 10. Therefore, only structures G, H, and I that were not excluded can be considered as candidates for subjects a1 to a3 recognized in Figure 4, and the correspondence between the subjects and structures can be investigated in the same way as in Figure 6.

In the above example, the field of view range in the horizontal direction is R1 and the field of view range in the vertical direction is R2, in which case a rectangular image as shown in Figure 4 is obtained. However, the shape of the field of view is not limited to this rectangular shape, and even if it is rectangular, it may be tilted around the optical axis as in the virtual image in Figure 6(b). Even in such cases, the control unit 42 can similarly determine whether each structure is within the field of view range.

In this way, the control unit 42 can recognize which of the structures G, H, I, X, Y, and Z each of the subjects recognized in Figure 4 corresponds to. In this case, the more accurately the items in the imaging information are recognized, the easier and more accurately the correspondence between each subject and each structure can be estimated.

The control unit 42 can also estimate the distance between each recognized subject and the imaging device 10 based on the focus state of each subject in the recognized image (Figure 4). In this case, when estimating the correspondence between each subject and each structure, this distance can be used in addition to the position information of the imaging device 10, as described above.

If the control information for the optical system 11 is included in the imaging information, the control unit 42 can use this control information to recognize the distance range in which the image in Figure 4 is in focus. Figure 9 shows an example in which the distance range from the imaging device 10 is S1. The distance range S1 corresponds to the depth of field of the optical system 11. Note that while Figure 9 is a plan view corresponding to Figure 5, in reality, the distance range corresponding to the depth of field can also be recognized in the vertical direction shown in Figure 8.

Figure 10 shows a case where the focus conditions for each subject are different from those in Figure 4. Here, subject a2 is in focus, resulting in high contrast at its edge (outline), while subjects a1 and a3 are out of focus, resulting in blurred edges. The control unit 42 can recognize this situation using well-known edge analysis techniques in the image (video) profile. Therefore, if the position of the imaging device 10 is determined as shown in Figure 9, the control unit 42 can estimate that subject a2 is structure H because structure H is the only object within the distance range S1 from the imaging device 10. Once the position of the imaging device 10 is determined and subject a2 is estimated to be structure H in this way, even if the orientation and field of view of the imaging device 10 are unknown, the control unit 42 can estimate the correspondence between the other subjects a1 and a3 and other structures, along with the orientation and field of view of the imaging device 10. If the depth of field is greater than S1 in Figure 9, then in addition to structure H, structures G and I could also be candidates for the in-focus subject. However, if the range of distance to the subject is recognized in this way, the structures that are candidates for the subject are further limited, allowing the above estimation to be performed more accurately.

Furthermore, rather than using the depth of field to determine the distance to the subject as described above, it is possible to more accurately determine the distance to the subject by using a distance measuring means that uses, for example, laser light. In this case, the control unit 15 of the imaging device 10 can include information about the distance to the subject in the video in the imaging information, and the control unit 42 of the video server system 40 can recognize this distance. This corresponds to the case in Figure 9 where S1 is very small (when the depth of field is small).

In this case, as shown in Figure 11, it is possible to recognize the distances (d1, d2, d3) between each subject in the image of Figure 4 and the imaging device 10. Meanwhile, the control unit 42 can recognize the distance between each structure in Figure 5 and the imaging device 10 from the position information and map information of the imaging device 10, as shown in Figure 12, and can recognize that the distances to structures G, H, and I are D1, D2, and D3, respectively. Under these conditions, it is possible to investigate the correspondence between each subject and each structure, and estimate this correspondence more easily and with higher accuracy.

In this way, if the distance to the subject can be recognized, the control unit 42 can estimate the correspondence between each subject and each structure particularly easily and with high accuracy. If the imaging device 10 is equipped with a distance measuring means, this distance can be included as one item of imaging information, and the video server system 40 can obtain this. However, as mentioned above, even if this distance is not included in the imaging information, it can be estimated from the focus status of the subject in the video.

Note that LiDAR (laser radar) is known as a device that emits pulsed laser light to obtain an optical image (takes an image) and also obtains distance information corresponding to the image. For this reason, LiDAR is particularly suitable for use as the imaging device 10. In this case, the distance to all locations in the image can be recognized, making it possible to estimate the correspondence between each subject and each structure particularly easily and with high accuracy.

As described above, the control unit 42 recognizes the correspondence between the subjects and structures in the video by analyzing the video based on the received video signal. If the map information includes identification information for each structure (e.g., building name, address, etc.), the control unit 42 can associate this with the subject and store it in the memory unit 44. This makes it particularly easy to manage the video for later use.

Furthermore, when analyzing the image as described above, if there is distortion in the image due to, for example, the characteristics of the optical system 11, this can be corrected before performing the above analysis, allowing for more accurate analysis.

Figure 13(a) shows the image corresponding to Figure 4 when such distortion is present. In this case, if the video server system 40 recognizes the characteristics of the imaging device 10 in advance, it can perform filtering on the video signal to obtain an image in which the distortion has been compensated for. If the parameters for this filtering process are stored in the memory unit 44 in advance, the control unit 42 can compensate for the distortion in the actual image such as Figure 13(a) and obtain the original image such as Figure 13(b). This compensated image can then be used to properly analyze the correspondence between the subject and the structure as shown in Figure 6. Note that instead of compensating for distortion in the actual image, a correction can be made to add this distortion to a virtual image obtained from map information, and an analysis such as that shown in Figure 6 can be performed.

As mentioned above, various types of virtual images, such as topographical maps and aerial photographs, are used as appropriate to compare with actual images. Furthermore, the images that are actually the subject of analysis may be daytime images or nighttime images. For this reason, when displaying on the display unit 45 as shown in FIG. 6, it is preferable to use images that have been adjusted for brightness, contrast, white balance, etc. for the images to be analyzed. In this case, such corrections can be made using the time included in the imaging information and the illuminance recognized by the illuminance sensor 20.

By using the imaging system 100 or video server system 40, which recognizes the correspondence between subjects and structures as described above, subjects in video footage can be recognized in real time as known structures. This makes it possible to perform operations such as the following:

Generally, the video captured by the imaging device 10 is not used as is for public broadcast (distribution), but rather the video is often processed and stored in the storage unit 44. This processing includes, for example, rendering (blurring, etc.) of specific subjects. Video server systems that perform this type of processing are described, for example, in Japanese Patent Application Laid-Open Nos. 2012-34218 and 2019-62381. As described above, if the subject in the video is identified as a known structure and its identification information is recognized, it becomes particularly easy to automatically perform this processing in the video server system 40. In this case, the imaging device 10 simply transmits the imaging information along with the video signal to enable the above operation. As described above, the imaging information is recognized only using the imaging device 10.

Furthermore, for example, when the imaging device 10 (and FPU device 30) is mounted on a helicopter or the like and photographed by a photographer, the helicopter or the like often only has the bare minimum of equipment installed. In such cases, when photographing is performed while the helicopter or the like is moving, the subject changes from moment to moment, and the photographer often cannot directly obtain map information such as that shown in Figure 5, and is often unable to recognize in real time the specific identity of the subject in the photographed video.

If the photographer is unable to properly identify the subject in real time, the photographer may not be able to properly capture the image. In contrast, in the above-described photography system 100, the video server system 40 can recognize the specific identity of the subject in real time, allowing the control unit 42 to issue appropriate instructions to the FPU device 30 located near the photographer.

Figure 14 shows an example of the image (a) and map information (b) in such a case. Here, the imaging device 10 is assumed to be located at position "C5" in Figure 5, and its horizontal field of view is assumed to be R3 in Figure 14(b). In this image (Figure 14(a)), the control unit 42 can recognize structures H, I, and X as subjects, as described above.

In this case, if the actual object to be photographed is structure G, structure G will not be included in the image (Figure 14(a)), and the control unit 42 can recognize this by recognizing structures H, I, and X as subjects. In this case, to photograph structure G, the control unit 42 can issue an instruction to the FPU device 30 (imaging device 10) to change the field of view to R4 in Figure 14(b), for example. Alternatively, the control unit 42 may instruct the imaging device 10 to move downward in Figure 14(b) without changing the field of view.

This instruction can be displayed as text information on the monitor of the imaging device 10 or the FPU device 30, for example. Alternatively, if the azimuth angle or elevation angle of the imaging device 10 is controlled by stage control, a control signal for this purpose may be sent from the video server system 40 to the FPU device 30 (imaging device 10). Furthermore, if the imaging device 10 is mounted on a device whose flight is controlled, such as a drone, a control signal for this device may be sent.

When the subject to be photographed is set in advance in this way, map information about the area around the subject can be stored in the memory unit 44 before the actual photographing begins, and the settings of the optical system 11 in the imaging device 10 (such as the lens to be used and the optical system parameters included in the imaging information) can be recognized in advance. The above analysis can be performed in real time as soon as photographing begins, allowing the subject to be photographed quickly.

Furthermore, the FPU device 30 must communicate wirelessly with an opposing FPU device that is installed in a building, radio tower, etc. and connected to the video server system 40, while facing it with high accuracy. If this building, radio tower, or other structure can be recognized as a subject as described above, instructions to adjust the direction of the antenna of the FPU device 30 can be given to the photographer, similar to the instructions given to the photographer regarding filming. In other words, the above-mentioned filming system 100 can also ensure smooth communication between the FPU device 30 and the video server system 40.

Figure 15(a) is a diagram showing the configuration of the imaging device 10 and FPU device 30 in such a case. Here, the FPU device 30 is equipped with a parabolic antenna 31 and a viewfinder 32. The antenna 31 has directivity as indicated by the arrow in the figure. The field of view of the imaging device 10 has a field of view range that generally follows the directivity of the antenna 31. In this state, the FPU device 30 is fixed to, for example, a camera platform, making it possible to adjust the orientation of the FPU device 30 (antenna 31) and imaging device 10.

In this state, the image captured by the imaging device 10 before the video signal is transmitted is assumed to be as shown in Figure 15(b). In this case, the FPU device 30 can display the image based on the video signal obtained from the imaging device 10 on the viewfinder (adjustment display unit) 32, similar to the display unit 45 in the video server system 40. Here, as in the above example, it is assumed that structures G, I, and H are recognized in this image. Furthermore, it is assumed that the opposing FPU device is installed above structure I.

Here, because the imaging device 10 is fixed to the FPU device 30, the FPU device 30 recognizes the imaging information and can display the orientation of the antenna 31 in the image as a point in the image. In Figure 15(b), this orientation is indicated by a cross mark M. This point needs to be aligned with the top of the structure I.

In this case, the control unit 42 can issue an instruction to change the orientation of the FPU unit 30 (antenna 31) in the viewfinder 32 so that the cross mark M overlaps the top of the structure I. This instruction can be issued by the FPU unit 30 (imaging device 10), as in the case above. Alternatively, even without issuing such an instruction, this adjustment can be made by the photographer looking at the display in Figure 15(b) and changing the orientation of the imaging device 10 while recognizing the positional relationship between the cross mark M and the structure. In other words, by performing the above analysis in real time, such antenna adjustments can be made efficiently.

In the above example, the above analysis is performed in real time on the video server system 40 side in order to optimize the shooting conditions and communication conditions. However, rather than performing this analysis in real time, the same analysis may be performed at a desired time after the video signal is stored in the memory unit 44. In this case, after this analysis process, it is possible to easily perform processing such as rendering on the video. If the imaging information includes the time of imaging, the correspondence between the video signal and the imaging information can be easily recognized, making it possible to easily perform such processing.

In the above example, the video server system 40 (control unit 42) performs the above-described analysis of the subject in the video. However, this analysis can be performed outside the video server system, for example, in the imaging device or FPU device, as long as map information is available. Alternatively, it can be performed in other components not shown in Figure 1. In other words, the analysis unit that performs the above-described analysis and the display unit that displays the video in Figure 6 and the like can be provided in any location in the components that make up this imaging system.

The present invention has been described above based on an embodiment. This embodiment is merely an example, and those skilled in the art will understand that various modifications are possible in the combination of the components, and that such modifications are also within the scope of the present invention.

10 imaging device 11 optical system 12 imaging element 13 video signal processing unit 14 video signal output unit 15 control unit (imaging device control unit)
16 Position information recognition unit 17 Altitude sensor 18 Direction sensor 19 Inclination sensor 20 Illuminance sensor 21 Time recognition unit 30 FPU device (transmission device)
31 Antenna 32 Viewfinder (adjustment display)
40 Video server system 41 Wireless communication unit (communication unit)
42 Control unit (analysis unit)
43 Network connection unit 44 Storage unit 45 Display unit 100 Photography systems a1 to a3 Subjects G, H, I, X, Y, Z Structure M Cross mark

Claims (7)

An imaging system using an imaging device that captures images and outputs video signals,
The imaging device is
an imaging information output unit that outputs imaging information that includes position information of the imaging device at the time of imaging and indicates a state of the imaging device at the time of imaging,
a communication unit that receives the video signal and the imaging information from the imaging device;
an analysis unit that recognizes a subject in an image obtained by the video signal, recognizes map information about the surroundings of the imaging device, including information about structures surrounding the imaging device, based on the position information, and recognizes a correspondence between the subject and the structures by comparing the image with the map information;
Equipped with
when a part of information other than the position information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates an imaging range for each of the parameters in the map information imaged by the imaging device, recognizes a correspondence between the subject and the structure in the imaging range, and determines that the value of the parameter when the subject and the structure most closely match corresponds to the value of the missing information;
An imaging system characterized in that the structure determined not to be recognized in the imaging range based on information other than the missing information is excluded in recognizing the correspondence relationship . The imaging system described in claim 1, further comprising a display unit that displays the image while indicating the correspondence between the subject and the structure. The imaging system described in claim 1 or 2, characterized in that the analysis unit recognizes in advance the object to be imaged by the imaging device, acquires the map information around the object, and compares the map information with the image based on the video signal obtained from the imaging device, thereby issuing instructions to the imaging device to control the imaging device so that the object is imaged. 3. The photographing system according to claim 1, wherein the photographing information includes any one of the altitude of the photographing device, the direction of the center of the field of view at the time of photographing, the tilt angle around the optical axis , the field of view range, and the distance to the photographed object. The photography system described in claim 1 or claim 2, characterized in that the analysis unit registers identification information of the structure recognized as the subject. the imaging device;
a video server system having the communication unit and the analysis unit;
a transmission device that receives the video signal and the imaging information from the imaging device and transmits them to the video server system by wireless communication via a directional antenna;
Equipped with
the imaging device is fixed to the transmission device;
The photographing system according to claim 1 or claim 2, further comprising an adjustment display unit that displays the orientation of the antenna in the image including the subject for which a correspondence relationship with the structure has been identified. A video server system that receives and processes video signals obtained by an imaging device,
a communication unit that receives, from the imaging device, imaging information that includes position information of the imaging device at the time of imaging and indicates a state of the imaging device at the time of imaging, together with the video signal;
an analysis unit that recognizes a subject in an image obtained by the video signal, recognizes map information about the surroundings of the imaging device, including information about structures surrounding the imaging device, from the position information, and recognizes a correspondence between the subject and the structures by comparing the image with the map information;
Equipped with
when a part of information other than the position information in the imaging information is missing, the analysis unit uses the missing information as a parameter, calculates an imaging range for each of the parameters in the map information imaged by the imaging device, recognizes a correspondence between the subject and the structure in the imaging range, and determines that the value of the parameter when the subject and the structure most closely match corresponds to the value of the missing information;
A video server system characterized in that the structure determined not to be recognized in the imaging range based on information other than the missing information is excluded in recognizing the correspondence .