JP7733286B2 - Imaging device - Google Patents

Description

The present invention relates to a system and method for recording and playing back an image showing location information and an image captured by a user as a single video.

In recent years, the advancement of digital still cameras and video cameras has made it extremely easy for users to take and record images, allowing them to record large amounts of video information. For example, if you go on a long trip and then look back at the footage you took after the trip, you may find yourself unable to remember which images were taken and where. In such cases, it is extremely convenient to view and organize the footage later if map information and the footage are combined in a single piece of video data. It is also extremely convenient for data taken with surveillance video cameras installed in multiple locations if the footage is combined with images showing location information in a single piece of video data.

A similar technology is driving recorders, which have recently begun to be installed in taxis and other vehicles. Driving recorders automatically determine the timing to take pictures and record situational data, including location information, as well as the captured data. Because automatic capture timing can result in the recording of large amounts of unnecessary data, technologies have been invented to prevent this. For example, Patent Document 1 uses a learning function to select the data to record, and after taking a picture, the user can input necessary/unnecessary information later, allowing the system to learn whether or not to take a picture.

Japanese Patent Application Laid-Open No. 2007-11907

However, because the footage is shot automatically, it may not be completely tailored to the user's needs, and it is not suitable for users to freely shoot their preferred footage along with map information.

One aspect of the present invention has been made in consideration of the above-mentioned problems, and provides an imaging device that can output a user's captured image in association with location information obtained from another device without performing complex processing.

One aspect of the present invention disclosed in this specification is an imaging device comprising: a wireless communication means for wirelessly acquiring, from an external device wirelessly connected to the imaging device, current location information of the external device acquired by the external device and information on a shooting instruction; an imaging means for performing a shooting process based on the information on the shooting instruction acquired by the wireless communication means; a recording means for correlating image data acquired by the imaging means by performing the shooting process based on the information on the shooting instruction with the current location information of the external device acquired by the wireless communication means and recording the associated image data on a recording medium; and a control means for controlling the wireless communication means, the imaging means, and the recording means, When outputting the image data to the outside in an environment where data can be transmitted from the imaging device to the outside, the imaging device has a first control mode in which the image data, which has been acquired by the imaging device by performing a shooting process based on information regarding shooting instructions acquired by the wireless communication device and recorded in association with current location information of the external device, is read from the recording medium and automatically output to the outside, and a second control mode in which the image data, which has been acquired by the imaging device by performing a shooting process based on information regarding shooting instructions acquired by the wireless communication device and recorded in association with current location information of the external device, is read from the recording medium and output to the outside at a timing set by a user.

According to one aspect of the present invention, it is possible to provide an imaging device that can output an image captured by a user in association with location information acquired from another device .

1 is a diagram showing the configuration of an image recording and reproducing system according to a first embodiment of the present invention; Flow diagram of the image recording and reproducing system according to the first embodiment 1 is a schematic diagram of stream data generated by the image recording and reproducing system according to the first embodiment; 1 is a diagram showing the configuration of an image recording and reproducing system according to a second embodiment of the present invention; Flow diagram of an image recording and reproducing system according to a second embodiment Stream data generated by the image recording and reproducing system of the second embodiment 10 is a diagram showing the configuration of an image recording and reproducing system according to a third embodiment of the present invention. Schematic diagram of a system for transferring stream data in a hotspot Thumbnail display example

The best mode for implementing the present invention is described below.

The following describes an embodiment of the image recording and playback system and method of the present invention, with reference to the accompanying drawings.

A first embodiment will now be described. In this embodiment, the operations from when a user issues a shooting instruction to when the shooting is actually performed will be described using Figures 1 to 3. Figure 1 illustrates an image recording and playback system in a first embodiment of the present invention. Video camera 1 includes an operation input unit 11, such as a button or touch panel, through which a user directly inputs various instructions, including shooting instructions; a control unit 12 that controls various units related to shooting; a captured image input unit 13 that inputs an image to be shot by the user, digitizes it, and outputs captured image data; an external image input unit 14 that inputs video from a navigation system 2, which will be described later; an image input arbitration unit 15 that arbitrates and outputs the order of the captured image data input from captured image input unit 13 and the navigation image data input from external image input unit 14; an image memory unit 16 that temporarily stores data output from image input arbitration unit 15; an image memory unit 16 that reads image data from image memory unit 16 and converts it into MPEG2 or H.264 format. The video camera 1 includes an image encoding unit 17 that compresses and encodes images using a method such as H.264 and outputs encoded image data; an audio input unit 18 that inputs audio from the subject being filmed by the user, digitizes it, and outputs audio data; an audio encoding unit 19 that compresses the audio data output by the audio input unit 18 using a method such as AC3 or Dolby Digital and outputs encoded audio data; a data generation unit 1A that multiplexes the encoded image data and encoded audio data and outputs stream data; a data recording unit 1B that records the stream data on a recording medium (described below); and a recording medium 1C such as a DVD, Blu-ray (registered trademark), hard disk, or semiconductor memory. The video camera 1 is connected to a navigation system 2 via an AV cable 3. The navigation system 2 has a navigation main unit 21 and a navigation display unit 22. The navigation main unit 21 has a navigation image output unit 211 that outputs navigation images. The navigation system 211 constantly outputs video signals to the navigation display unit 22 and also outputs signals to the AV cable 3 connected to an external video terminal (not shown).

Figure 2 shows the operational flow of each part in Example 1.

The shooting procedure in the image recording and playback system of Example 1 will now be described. The user issues a shooting instruction to the operation input unit 11 using buttons or switches on the video camera 1. The operation input unit 11 issues a shooting start instruction to the control unit 12. The control unit 12 issues a start instruction to the captured image input unit 13, the audio input unit 18, and the external image input unit 14. In response to the start instruction, the external image input unit 14 receives a video signal constantly output from an external video terminal of the navigation system 2 or the like via the AV cable 3, generates navigation image data, and outputs it to the image input arbitration unit 15. The captured image input unit 13 inputs an image of the subject the user is shooting, digitizes it to generate captured image data, and outputs it to the image input arbitration unit 15. The audio input unit 18 inputs the audio of the subject the user is shooting, digitizes it to generate audio data, and outputs it to the audio encoding unit 19. The navigation image data and captured image data input to the image input arbitration unit 15 are temporarily stored in an area of the image memory 16, which is set to be encoded in this order. That is, the navigation image data is temporarily stored in one area of the image memory unit 16, from which the downstream image encoding unit 17 reads the first image data when processing the first frame, and the captured image data is temporarily stored in another area of the image memory unit 16, from which the image data for the second and subsequent frames is read when processing the second and subsequent frames. The navigation image data and the captured image data temporarily stored in the image memory unit 16 are read out as image data to the image encoding unit 17 in this order. The image encoding unit 17 compresses and encodes the image data. Here, the image encoding unit 17 compresses and encodes the first image, the navigation image data, as an I-picture of H.264, by compressing and encoding only data within one frame. The captured image data following the navigation image data is similarly compressed and encoded as an I-picture, by compressing and encoding the first frame's worth of image data. The subsequent images are compressed and encoded, for example, as P-pictures or B-pictures predicted from images in the forward or backward direction, or as I-pictures compressed and encoded only data within one frame, or as a mixture of these, to generate encoded image data, which is output to the data generating unit 1A. Furthermore, the image encoding unit 17 changes the processing speed to approximate real-time operation by, for example, coarsely quantizing the data to reduce the amount of code and make processing easier, or by temporarily increasing the clock frequency. Meanwhile, the audio encoding unit 19 compresses and encodes the audio data input thereto to generate encoded audio data, which is then output to the data generating unit 1A. The data generating unit 1A multiplexes the encoded image data and the encoded audio data to generate video stream data. An example of the stream data is shown in FIG. 3 . In FIG. 3 , the stream data is shown from top to bottom in the order of the images that have been decoded and expanded. The first image represents a navigation screen, compressed and encoded as an I-picture and multiplexed so that no audio data is present. The second, third, and fourth images represent captured image data, compressed and encoded as an I-picture, the third as a B-picture, and the fourth as a P-picture, and the audio data is multiplexed using the encoded audio data. The stream data generated by the data generation unit 1A is output to the data recording unit 1B, which then records the stream data on the recording medium 1C.

To play back the stream data recorded in this manner on the video camera 1, a picture analysis unit (not shown) is provided that analyzes the picture type of the stream data. If the stream data contains two or more consecutive I pictures, the first I picture is output to the display unit of the video camera 1 (not shown), and the same image is displayed for a predetermined period of time, after which the next I picture is output to the display unit. If the subsequent pictures are not I pictures, they are output one after another to the display unit, as with normal video data playback. Note that, for stream data containing two or more consecutive I pictures, the first I picture is output to the display unit of the video camera 1 (not shown), and the same image is displayed for a predetermined period of time; however, the next image may be displayed based on a user operation, such as displaying the same image until the user presses the play button.

Furthermore, in video compression/encoding methods that have a mechanism for embedding data within a stream to stop stream playback, such as the Still command of the DVD-Video standard, playback on devices other than the video camera 1 can be supported by, for example, dividing the VOBU (Video Object Unit) into pictures for the navigation screen and pictures of captured images from the second screen onwards, and inserting a VOBU Still command in the data generation unit 1A.

As described above, in addition to the means for inputting normal captured images into the video camera, a means for inputting images from another video input source is provided, and an image input arbitration unit is further provided that arbitrates and imports the video data from these two input means and stores it in memory. By using the above-mentioned other video input source as the video output terminal of the car navigation system, the car navigation screen (map information) and images captured by the user can be imported into memory together. By compressing and encoding the data in this memory as a moving image, the map information and images captured by the user can be processed as a single moving image.

Furthermore, in this embodiment, the navigation system 2 is connected externally to the video camera 1, but the same effect can be achieved even if the video camera 1 and navigation system 2 are integrated into one device.

The stream data recorded on the recording medium 1C may also be automatically transferred to a server on a network via a wireless LAN environment (hotspot) installed around town. Figure 8 shows a schematic diagram of a system for transferring the stream data to a server in a hotspot environment. The video camera 1 has a data reading unit 1E that reads the stream data from the recording medium 1C, and a wireless communication unit 1F that wirelessly transmits the stream data to the network 4. The stream data output from the wireless communication unit 1F is transferred to a server 5 connected to the network 4.

In this embodiment, the data generation unit 1A simply generates stream data. However, it may also receive current location information from the navigation system 2 separately from the navigation image data, generate stream data associated with the current location information, output the stream data to the data recording unit 1B, and record it on the recording medium 1C. In this case, when generating a thumbnail screen for the stream data recorded on the recording medium 1C, the current location information may be used as a clue to create a thumbnail display screen in combination with a map screen, as shown in FIG. 9. In the example of FIG. 9, a circle indicates the location on the map corresponding to the location information associated with the stream data, and the first image captured by the stream data is displayed in association with the circle. Furthermore, when the video camera 1 has multiple recording media, such as a hard disk and a DVD, and the video camera 1 normally records data on the hard disk, and when the user selects, edits, and records the stream data on a DVD, a title screen like the one shown in FIG. 9 may be automatically created and recorded on the DVD.

In addition, in this embodiment, the user operates buttons and switches on the video camera 1 to issue shooting instructions from the operation input unit 11, but the navigation system 2 and video camera 1 may be provided with communication means such as USB, Bluetooth, or wireless LAN, and the user may set the location where they wish to shoot in the navigation system 2 in advance. When the user reaches the set location, the navigation system 2 outputs a shooting instruction to the operation input unit 11 of the video camera 1, and the operation input unit 11 outputs a shooting start instruction to the control unit 12, thereby starting shooting.

Furthermore, by replacing the navigation system 2 and AV cable 3 with a medium on which map image data is pre-stored, such as a USB flash memory or SD card, and a reading device that reads the map image data from the medium, and by transferring data to a server on a network in a hotspot environment as shown in Figure 8, and by including a sending means that reads compressed image data recorded on the recording medium, packetizes the compressed image data at a predetermined timing set by the user, and sends it to the network, the same effect can be achieved even in video cameras installed in multiple locations for surveillance purposes, etc. In this case, the wireless communication unit 1F may be replaced with a function that communicates via a wired LAN.

Now, a second embodiment will be described. Figure 4 illustrates an image recording and playback system according to a second embodiment of the present invention. This embodiment differs from the first embodiment in that, in addition to the components shown in Figure 1, video camera 1 includes an external audio input unit 181 that inputs audio from navigation system 2, an audio input arbitration unit 182 that arbitrates and outputs the order of audio data input from audio input unit 18 and navigation audio data input from external audio input unit 181, an audio memory unit 183 that temporarily stores data output from audio input arbitration unit 182, and an audio activation unit 1D that transmits an audio activation instruction to navigation system 2 via, for example, infrared communication to activate audio. Furthermore, navigation main unit 21 of navigation system 2 includes, in addition to the components shown in Figure 1, a navigation audio activation unit 213 that receives an audio activation instruction from audio activation unit 1D of video camera 1 via, for example, infrared communication, and issues an activation instruction to output navigation audio, and a navigation audio output unit 212 that outputs navigation audio in response to the activation instruction.

Figure 5 shows the operational flow of each part in Example 2.

The shooting procedure in the image recording and playback system of Example 2 will now be described. The user issues a shooting instruction to the operation input unit 11 of the video camera 1. This shooting instruction may be, for example, an operation to activate a voice message from the navigation system 2 indicating location information, such as pressing the "Current Location" button on the navigation system. The operation input unit 11 issues a shooting start instruction to the control unit 12. The operation input unit 11 also issues a voice activation instruction to the voice activation unit 1D. The control unit 12 issues activation instructions to the captured image input unit 13, the voice input unit 18, the external image input unit 14, and the external voice input unit 181. In response to the activation instruction, the external image input unit 14 captures a video signal constantly output from the navigation system 2 via an external video terminal or the like, generates navigation image data, and outputs it to the image input arbitration unit 15. In this case, the navigation image data is data of one or more pictures for a certain period of time set in advance by the user, etc. The captured image input unit 13 inputs an image of the subject the user is shooting, digitizes it, generates captured image data, and outputs it to the image input arbitration unit 15. Meanwhile, in response to a voice activation instruction issued to the voice activation unit 1D by the operation input unit 11, the voice activation unit 1D issues a voice activation instruction to the navigation voice activation unit 213 of the navigation system 2 via infrared communication or the like. This voice activation instruction may be output from the voice activation unit in the form of a signal that is the same as a signal output to the navigation system 2 from a remote control (not shown) attached to the navigation system when a "current location" button or the like is pressed, which is a typical navigation function. Upon receiving the voice activation instruction, the navigation voice activation unit 213 issues an activation instruction to the navigation voice output unit 212. In response to the activation instruction, the navigation voice output unit outputs a voice such as "Your current location is near Yoshida-cho, Totsuka-ku, Yokohama City," and outputs it from an external voice terminal or the like. In response to the activation instruction from the control unit 12, the external voice input unit 181 captures the navigation voice output from the navigation system 2 via an external voice terminal or the like, generates navigation voice data, and outputs it to the voice input arbitration unit 15. In this case, the navigation voice data is data for a certain period of time that is preset by the user or the like. This set time may be the same as the time when the external image input unit 14 captures the data. The audio input unit 18 inputs audio of the subject being photographed by the user, digitizes it, and outputs the audio data to the audio encoding unit 19. The navigation image data and the photographed image data input to the image input arbitration unit 15 are temporarily stored in areas of the image memory 16, which are set to be encoded in this order. That is, the navigation image data is temporarily stored in one area of the image memory 16 that is read by the downstream image encoding unit 17 when processing the first image data, and the photographed image data is temporarily stored in another area of the image memory 16 that is read by the downstream image encoding unit 17 when processing the second and subsequent image data. The navigation image data and the photographed image data temporarily stored in the image memory 16 are read as image data by the image encoding unit 17 in this order. The image encoding unit 17 compresses and encodes the image data. Here, the image encoding unit 17 compresses and encodes the navigation image data, which is the first image, using only data within one screen, for example, as an I-picture according to H.264. The captured image data following the navigation image data is compressed and encoded as an I-picture, with the first frame's worth of image data being compressed and encoded as an I-picture. Subsequent images are compressed and encoded as, for example, P-pictures or B-pictures, which are predicted from previous or subsequent images, or as I-pictures, which are compressed and encoded using only data within one frame, or as a mixture of these. The resulting encoded image data is then output to the data generator 1A. Meanwhile, the navigation voice data and the voice data input to the audio input arbitration unit 182 are temporarily stored in an area of the voice memory 183, which is set to be encoded in this order. The navigation voice data and the voice data temporarily stored in the voice memory 183 are read out in this order by the voice encoder 19 as mixed voice data. The mixed voice data input to the voice encoder 19 is compressed and encoded by the voice encoder 19 to generate coded voice data, which is then output to the data generator 1A. The data generator 1A multiplexes the coded image data and coded voice data to generate video stream data. FIG. 6 shows a schematic diagram (one example) of stream data. In FIG. 6, the stream data is shown in the order of images that have been decoded and expanded, from top to bottom. The top three images show navigation screens, compressed and encoded as I, B, and P pictures, from top to bottom, with the audio data multiplexed with data corresponding to external audio data from the encoded audio data. The bottom three images show captured image data, compressed and encoded as I, B, and P pictures, from the fourth image onwards, with the audio data multiplexed with data corresponding to the audio data from the encoded audio data. Note that while the top three images in FIG. 6 show the same image, they may each be a different navigation image. The stream data generated by data generation unit 1A is output to data recording unit 1B, which then records the stream data on recording medium 1C.

The above process achieves the same effect as in Example 1.

In addition, in this embodiment, the navigation image data is data for a certain period of time set in advance by the user, etc. However, it is also possible to provide communication means such as USB, Bluetooth, or wireless LAN in the navigation system 2 and video camera 1, and when the navigation audio ends, the navigation system 2 outputs a navigation audio end notification to the external audio input unit 181 of the video camera 1, and the external audio input unit 181 determines the end of the data based on this end notification.

Example 3 will now be described. The difference from Example 1 is that, whereas in Example 1, video camera 1 and navigation system 2 were connected by AV cable 3, video camera 1 is connected to C-side wireless communication means 32, and navigation system 2 is connected to N-side wireless communication means 31. C-side wireless communication means 32 and N-side wireless communication means 31 communicate with each other and exchange data using wireless communication protocols such as infrared communication, wireless LAN, and Bluetooth (registered trademark).

The shooting procedure in the image recording and playback system of Example 3 is the same as Example 1, except that whereas in Example 1 the navigation image is exchanged from the navigation image output unit 211 to the external image input unit 14 via an AV cable, the navigation image is exchanged via the N-side wireless communication means 31 and the C-side wireless communication means 32.

The above process achieves the same effect as in Example 1.

1 Video camera 11 Operation input unit 12 Control unit 13 Captured image input unit 14 External image input unit 15 Image input arbitration unit 16 Image memory unit 18 Audio input unit 181 External audio input unit 182 Audio input arbitration unit 183 Audio memory unit 1D Audio activation unit 2 Navigation 21 Navigation main unit 211 Navigation image output unit 212 Navigation audio output unit 213 Navigation audio activation unit 22 Navigation display unit 3 AV cable

Claims (18)

An imaging device,
a wireless communication means for wirelessly acquiring, from an external device connected to the imaging device wirelessly, current location information of the external device acquired by the external device and information related to a shooting instruction;
an imaging unit that performs imaging processing based on information about the imaging instruction acquired by the wireless communication unit;
a recording means for recording image data acquired by the imaging means by performing an imaging process based on the information relating to the imaging instruction and current location information of the external device acquired by the wireless communication means in association with each other on a recording medium;
a control means for controlling the wireless communication means, the imaging means, and the recording means;
Equipped with
The control means
When the image data is output to the outside in an environment where data can be transmitted from the imaging device to the outside,
a first control mode in which image data acquired by the imaging means by performing an image capture process based on information relating to an image capture instruction acquired by the wireless communication means and recorded in association with current location information of the external device is read from the recording medium and automatically output to an external device;
a second control mode in which image data acquired by the imaging means by performing an image capture process based on information related to an image capture instruction acquired by the wireless communication means and recorded in association with current location information of the external device is read from the recording medium and output to an external device at a timing set by a user;
An imaging device comprising: 2. The imaging device according to claim 1,
The imaging device is characterized in that the wireless communication means acquires current location information of the external device when acquiring information related to the shooting instruction. 3. The imaging device according to claim 1,
In the first control mode, the image capturing apparatus outputs the image data to an external device so as to transfer the image data to a server on a network. 3. The imaging device according to claim 1,
In the second control mode, the image data is output to the outside so as to be transferred to a server on a network. 3. The imaging device according to claim 1,
In the first control mode, the image data is output to the outside together with a thumbnail of the image data. 3. The imaging device according to claim 1,
In the second control mode, the image data is output to the outside together with a thumbnail of the image data. 3. The imaging device according to claim 1,
The imaging device is characterized in that the image data is video data. An imaging device,
a first wireless communication means for wirelessly acquiring, from an external device wirelessly connected to the imaging device, current location information of the external device acquired by the external device;
an imaging unit that performs imaging processing based on information regarding an imaging instruction from the external device that is wirelessly connected to the imaging device;
a recording means for recording image data acquired by the imaging means by performing an imaging process based on information relating to the imaging instruction and current location information of the external device acquired by the first wireless communication means in association with each other on a recording medium;
a second wireless communication means different from the first wireless communication means, which outputs the image data wirelessly to an external device;
a control unit that controls the first wireless communication unit, the second wireless communication unit, the imaging unit, and the recording unit;
Equipped with
The control means
When the image data is output to the outside in an environment where data can be transmitted from the imaging device to the outside via the second wireless communication means,
a first control mode in which image data acquired by the imaging means by performing an image capturing process based on the information relating to the image capturing instruction and recorded in association with current position information of the external device is read from the recording medium and automatically output to an external device;
a second control mode in which image data acquired by the imaging means by performing an image capturing process based on the information related to the image capturing instruction and recorded in association with current location information of the external device is read from the recording medium and output to an external device at a timing set by a user;
An imaging device comprising: 9. The imaging device according to claim 8,
The imaging device further comprises acquiring current location information of the external device when acquiring information relating to a shooting instruction from the external device. 10. The imaging device according to claim 8,
The imaging device, wherein the second wireless communication means, in the first control mode, outputs the image data to an external device so as to transfer the image data to a server on a network. 10. The imaging device according to claim 8,
The imaging device, wherein the second wireless communication means, in the second control mode, outputs the image data to an external device so as to transfer the image data to a server on a network. 10. The imaging device according to claim 8,
The imaging device is characterized in that the first wireless communication means is a means for performing Bluetooth communication. 10. The imaging device according to claim 8,
The imaging device, wherein the second wireless communication means outputs the image data together with a thumbnail of the image data to an external device in the first control mode and the second control mode. 10. The imaging device according to claim 8,
The imaging device is characterized in that the image data is video data. 2. The imaging device according to claim 1,
In the first control mode, the imaging device automatically outputs the image data to an external device after performing a photographing process based on information related to a photographing instruction acquired by the wireless communication means. 2. The imaging device according to claim 1,
In the first control mode,
when information regarding a first photographing instruction is acquired in an environment in which data can be transmitted from the imaging device to an external device, performing a photographing process based on the acquired information regarding the first photographing instruction, and when first image data acquired by the imaging means is recorded on the recording medium in association with current location information of the external device, automatically outputting the first image data associated with the current location information to an external device;
An imaging device characterized in that, when information regarding a second shooting instruction different from the first shooting instruction is acquired in an environment in which data can be transmitted from the imaging device to the outside, the imaging device performs a shooting process based on the information regarding the acquired second shooting instruction, and when second image data different from the first image data acquired by the imaging means is recorded in association with current location information of the external device, the second image data associated with the current location information is automatically output to the outside. 2. The imaging device according to claim 1,
In the first control mode, the image capturing apparatus automatically outputs the image data to an external device every time a photographing process is performed based on information regarding a photographing instruction acquired by the wireless communication means. 2. The imaging device according to claim 1,
In the first control mode, which is different from the second control mode, the image data is automatically output to the outside after a photographing process based on information regarding a photographing instruction acquired by the wireless communication means, unlike the second control mode in which the image data is output to the outside at the user's timing.