JP6887263B2 - In-vehicle systems, in-vehicle devices, programs and methods - Google Patents

Description

The present invention relates to in-vehicle systems, in-vehicle devices, programs and methods.

Conventionally, when boarding a vehicle and moving to a sightseeing spot or the like, the scenery outside the vehicle may be imaged from the inside of the vehicle.

In a vehicle traveling in a scenic area, when a passenger other than the person driving the vehicle takes an image so that the passenger and an object to be imaged such as a mountain outside the vehicle are included in one image. There is.

Since the in-vehicle device having a navigation function has map information on which the vehicle is traveling, it can be determined that the vehicle is traveling in a scenic area outside the vehicle such as a sightseeing spot.

For example, Patent Document 1 proposes a video recording device for a vehicle that records a video outside the vehicle when the vehicle is traveling within a predetermined distance from a tourist spot.

Japanese Unexamined Patent Publication No. 2011-28651

However, the technique disclosed in Patent Document 1 described above does not disclose that a passenger in a vehicle and an object to be imaged outside the vehicle are simultaneously imaged.

An object of the present specification is to provide an in-vehicle system, an in-vehicle device, a program, and a method capable of photographing a passenger on a vehicle and an object to be imaged outside the vehicle.

According to the in-vehicle system disclosed in the present specification, the positions of the image pickup unit that acquires an image and the image pickup object outside the vehicle are acquired, and based on the positional relationship between the optical axis direction of the image pickup unit and the image pickup object outside the vehicle. A processing unit that determines whether the passenger and the imaging object can be imaged, and if the imaging is possible, causes the imaging unit to capture an image including the passenger and the imaging object, and an output unit that outputs the captured image. , Equipped with.

According to the in-vehicle device disclosed in the present specification, the passenger and the passenger and the image pickup object outside the vehicle are obtained based on the positional relationship between the optical axis direction of the imaging unit that acquires the position of the image pickup object outside the vehicle and the image pickup object outside the vehicle. It is provided with a processing unit that determines whether or not the image-imaging object can be imaged, and if the image can be imaged, causes the image pickup unit to image an image including the passenger and the image-imaging object, and an output unit that outputs the captured image.

Further, according to the program disclosed in the present specification, it is based on the acquisition of the position of the image pickup object outside the vehicle, the optical axis direction of the image pickup unit that acquires the image, and the positional relationship between the image pickup object outside the vehicle. The processor is made to determine whether or not it is possible to capture an image including the passenger and the object to be imaged, and to capture an image of the passenger and the object to be imaged by using the imaging unit when the image can be captured.

Further, according to the method disclosed in the present specification, it is based on the acquisition of the position of the image pickup object outside the vehicle, the optical axis direction of the image pickup unit for acquiring the image, and the positional relationship between the image pickup object outside the vehicle. It is provided to determine whether or not an image including the passenger and the object to be imaged can be captured, and to capture an image including the passenger and the object to be imaged by using the imaging unit when the image can be taken.

According to the in-vehicle system disclosed in the present specification described above, it is possible to image a passenger in a vehicle and an object to be imaged outside the vehicle.

According to the in-vehicle device disclosed in the present specification described above, it is possible to image a passenger in a vehicle and an object to be imaged outside the vehicle.

Further, according to the program disclosed in the present specification described above, it is possible to image a passenger on a vehicle and an object to be imaged outside the vehicle.

Further, according to the method disclosed in the present specification described above, it is possible to image a passenger in a vehicle and an object to be imaged outside the vehicle.

It is a figure which shows one Embodiment of the system disclosed in this specification. It is a hardware block diagram of the vehicle-mounted device disclosed in this specification. It is a figure explaining the imaging object table. It is a figure explaining the optical axis direction of the image pickup apparatus arranged in a vehicle. It is a figure explaining that it determines whether it is possible to image a passenger and an object to be imaged. It is a figure explaining the image pickup possible area, the image pickup recommended area, and the image pickup non-recommendation area. It is a figure explaining the storage part of a server. It is a sequence diagram explaining the operation of a system. It is a figure which shows the screen which selects the imaging area. It is a figure which shows the screen which selects the image pickup mode. It is a flowchart (the 1) explaining the operation of an in-vehicle device. It is a flowchart (2) explaining the operation of an in-vehicle device. It is a figure which shows the screen which notifies the imaging. It is a figure which shows the image which image | photographed the passenger and the image-catching object. It is a figure which shows the other form of an in-vehicle device.

Hereinafter, a preferred embodiment of the system disclosed in the present specification will be described with reference to the drawings. However, the technical scope of the present invention is not limited to those embodiments, but extends to the inventions described in the claims and their equivalents.

FIG. 1 is a diagram showing an embodiment of the system disclosed in the present specification.

The system 1 of the present embodiment includes an in-vehicle device 10 mounted on a vehicle and a server 40 that is communicably connected to the in-vehicle device 10 via a network N.

The in-vehicle device 10 obtains an imaging position capable of capturing an image of a passenger on the vehicle and an object to be imaged outside the vehicle on a route on which the vehicle on which the in-vehicle device 10 is mounted travels. The image is taken so as to be included in one image.

The server 40 stores the image pickup object information including the position information of the image pickup object. The server 40 transmits the image pickup target information to the vehicle-mounted device 10 via the network N. The in-vehicle device 10 obtains an imaging position capable of imaging the passenger and the imaging object outside the vehicle on the route to the destination based on the imaging object information received from the server 40.

Examples of the object to be imaged include natural objects such as the sea, mountains or valleys, and artificial landmarks such as towers or buildings. In the server 40, the image target object information is updated at any time.

Examples of the vehicle on which the in-vehicle device 10 is mounted include a vehicle having four or more wheels such as an automobile, a motorcycle and a light vehicle such as a bicycle. In this embodiment, the in-vehicle device is mounted on an automobile.

Next, the in-vehicle device 10 will be described in detail below.

FIG. 2 is a hardware configuration diagram of the in-vehicle device 10.

The in-vehicle device 10 includes a processing unit 11, a storage unit 12, a display unit 13, an operation unit 14, an acoustic output unit 15, a communication unit 16, and image pickup devices 20a to 20d.

The processing unit 11 has one or more processors and peripheral circuits. The processing unit 11 controls each hardware component of the in-vehicle device 10 and performs various processes according to a predetermined program 12a stored in the storage unit 12 in advance, and temporarily stores the data generated during the process. The storage unit 12 is used.

The processing unit 11 includes an optical axis direction determination unit 11a, an image pickup target setting unit 11b, a navigation unit 11c, and an image pickup determination unit 11d.

Each of these units included in the processing unit 11 is, for example, a functional module realized by a computer program running on the processor included in the processing unit 11. Each of these units included in the processing unit 11 may be mounted on the in-vehicle device 10 as a separate circuit. The explanation of each part will be described later.

The storage unit 12 may have a semiconductor memory such as a random access memory (RAM) or a read-only memory (ROM), or a non-volatile memory such as a magnetic disk or a flash memory. Further, the storage unit 12 may use a disk drive (not shown) as a drive capable of reading a non-temporary storage medium for storing the predetermined program 12a.

The storage unit 12 stores a predetermined program 12a, map data 12b, image data 12c, an image pickup target table 12d, an image pickup recommended area table 12e, and an image pickup non-recommended area table 12f.

FIG. 3 is a diagram illustrating a table of objects to be imaged.

In the imaging object table 300, the imaging object identification information column 301 in which the identification information for identifying the imaging object is registered, the name column 302 in which the name of the imaging object is registered, and the position information of the imaging object are registered. It has a position information column 303 to be displayed. The position information of the imaging object registered in the position information column 303 may be represented by one point in the coordinates on the map, or may be represented as a region having a predetermined area. The position information of the object to be imaged is represented by a two-dimensional map coordinate system representing the route of the vehicle in the map data.

The map data 12b, the image data 12c, the recommended imaging region table 12e, and the non-recommended imaging region table 12f will be described later.

The display unit 13 is arranged on, for example, the center console, and can be controlled by the processing unit 11 to display various information accompanying the operation of the in-vehicle device 10. As the display unit 13, for example, a liquid crystal display can be used.

The operation unit 14 is operated by the passenger and can input the operation. The in-vehicle device 10 can use, for example, a touch panel integrally formed with the display unit 13 as the operation unit 14.

The sound output unit 15 has a speaker (not shown) and is controlled by the processing unit 11 to output sound.

The communication unit 16 transmits / receives information to / from the server 40 via the network N by using wireless communication. The communication unit 16 connects to the network N via a base station in accordance with a predetermined communication standard such as 3GPP (Third Generation Partnership Project) or LTE (Long Term Evolution). Further, the communication unit 16 communicates with the image pickup devices 20a to 20d. Further, the communication unit 16 connects to the in-vehicle network (Controlled by Controller Area Network: CAN) mounted on the vehicle. For example, the processing unit 11 uses the communication unit 16 to receive a vehicle signal indicating the speed of the vehicle via the CAN. The communication unit 46 may have a communication circuit and a communication line or an antenna for transmitting and receiving.

Further, in the vehicle-mounted device 10, position information indicating the position of the vehicle is input from the GPS information receiving unit 30 mounted on the vehicle. The GPS information receiving unit 30 receives the GPS radio wave transmitted by the GPS artificial satellite, obtains the position of the vehicle, and outputs the position information of the vehicle to the in-vehicle device 10.

The navigation unit 11c of the processing unit 11 has a navigation function. The navigation unit 11c is a functional module realized by executing a program stored in the storage unit 12 using the map data 12b also stored in the storage unit 12, and is a current vehicle acquired from the GPS information receiving unit 30. Route guidance to the destination is provided based on the position of. The processing unit 11 generates a map image and voice guidance information showing a map around the vehicle based on the map data 12b stored in the storage unit 12 and the position of the vehicle. Further, when the destination is set by the passenger, the processing unit 11 derives a route from the current position of the vehicle to the destination and superimposes the route on the map image. The map image generated by the processing unit 11 is displayed on the display unit 13. The voice guidance information generated by the navigation unit 11c is output from the sound output unit 15.

Next, the imaging devices 20a to 20d will be described in detail below.

The imaging devices 20a to 20d include an imaging unit 21 and an optical axis adjusting unit 22.

The imaging unit 21 has a predetermined optical axis direction, and the optical axis adjusting unit 22 can be used to set the optical axis direction so as to image the outside of the vehicle including the passengers inside the vehicle. The imaging unit 21 has a viewing angle in a predetermined range centered on the optical axis direction, and can image a subject within the viewing angle range.

The imaging unit 21 has, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Sensor) image sensor. The image pickup unit 21 takes an image of the subject, generates image data, and transmits the generated image data to the in-vehicle device 10. The processing unit 11 of the in-vehicle device 10 stores the received image data 12c in the storage unit 12. Further, the processing unit 11 of the in-vehicle device 10 can output the received image data 12c to the display unit 13 and display it.

The optical axis adjusting unit 22 is operated by a occupant of the vehicle to adjust the optical axis direction of the imaging unit 21 and the enlargement ratio of the image. While the image pickup unit 21 is capturing the subject, the passenger of the vehicle uses the optical axis adjustment unit 22 while checking the image of the display unit 13 to obtain the optical axis direction of the image pickup unit 21 and the enlargement ratio of the image. To adjust. The optical axis adjusting unit 22 may be performed by using the operating unit 14 of the in-vehicle device 10.

The optical axis adjusting unit 22 transmits to the in-vehicle device 10 the moving amount information in the optical axis direction indicating the moving amount in the optical axis direction moved by the adjustment with reference to the optical axis direction of the imaging unit 21 in the initial state. The optical axis direction determining unit 11a of the processing unit 11 of the in-vehicle device 10 determines the optical axis direction of the imaging unit 21 based on the received information on the amount of movement in the optical axis direction of the imaging unit 21 in a vehicle coordinate system fixed to the vehicle. To decide on. That is, the optical axis direction of the imaging unit 21 is represented in the vehicle coordinate system. The vehicle coordinate system will be described later with reference to FIG.

FIG. 4 is a diagram for explaining the optical axis direction of the image pickup device arranged in the vehicle.

Four image pickup devices 20a to 20d are arranged in the passenger compartment of the vehicle 50.

The image pickup unit 21 of the image pickup device 20a is set in the optical axis direction A1 so as to image the passenger 51a seated in the driver's seat and the outside of the vehicle. The image pickup apparatus 20a is arranged, for example, inside the A pillar (not shown) on the left side of the vehicle 50.

The image pickup unit 21 of the image pickup device 20b is set in the optical axis direction A2 so as to image the passenger 51b seated in the passenger seat and the outside of the vehicle. The image pickup apparatus 20b is arranged, for example, inside the A pillar (not shown) on the right side of the vehicle 50.

The image pickup unit 21 of the image pickup device 20c is set in the optical axis direction A3 so as to image the passenger 51c seated behind the driver's seat and the outside of the vehicle. The image pickup apparatus 20c is arranged, for example, behind the driver's seat.

The image pickup unit 21 of the image pickup device 20d is set in the optical axis direction A4 so as to image the passenger 51d seated behind the passenger seat and the outside of the vehicle. The image pickup apparatus 20d is arranged, for example, behind the passenger seat.

The optical axis direction A1 of the image pickup apparatus 20a is represented by the vehicle coordinate system S fixed to the vehicle. The optical axis directions A2 to A4 of the other imaging devices 20b to 20d are also represented in the vehicle coordinate system S in the same manner.

In the present embodiment, the vehicle coordinate system S is a two-dimensional Cartesian coordinate system having an origin O fixed to the vehicle and X-axis directions and Y-axis directions. The vehicle coordinate system S may be a three-dimensional coordinate system, and in this case, the optical axis directions A1 to A4 are also represented by the three-dimensional coordinate system.

Next, whether or not the image pickup determination unit 11d of the processing unit 11 of the in-vehicle device 10 can image the passenger and the image pickup object based on the positional relationship between the optical axis direction of the image pickup unit 21 and the image pickup object outside the vehicle. The process of determining whether or not is described below.

FIG. 5 is a diagram for explaining whether it is possible to image the passenger and the object to be imaged.

The image pickup determination unit 11d is the passenger and the passenger at the image pickup position where the optical axis direction A1 of the image pickup device 20a intersects the position of the image pickup object 60 on the path 70 on which the vehicle 50 on which the in-vehicle device 10 is mounted travels. It is determined that the object to be imaged can be imaged. As a result, the passenger and the object to be imaged can be imaged at the most suitable timing for imaging. Here, when the position information of the image pickup object 60 is represented as an area having a predetermined area on the map, it is determined that the image can be taken in the area on the predetermined route on which the vehicle 50 travels. May occur.

The Y-axis direction of the vehicle coordinate system S coincides with the traveling direction of the vehicle in the two-dimensional map coordinate system representing the route to the destination. For example, the Y-axis direction of the vehicle coordinate system S may coincide with the direction in which the route on which the vehicle 50 is traveling extends. The image pickup determination unit 11d converts the optical axis direction A1 of the image pickup apparatus 20a represented in the vehicle coordinate system S into the representation of the map coordinate system. Then, the image pickup determination unit 11d obtains the position on the vehicle path 70 when the optical axis direction A1 of the image pickup device 20a represented by the map coordinate system intersects the position of the image pickup object 60 as an image pickup possible region. .. Specifically, the image pickup determination unit 11d examines whether or not a straight line extending from the position of the image pickup device 20a of the vehicle 50 in the optical axis direction A1 intersects with the image pickup object 60 on the map. The required imageable region may be a point on the path 70, or may be a part of a predetermined range on the path 70.

As described above, the optical axis direction A1 of the image pickup device 20a is set to take an image of the outside of the vehicle including the passenger 51a in the driver's seat. The passenger 51a and the image pickup object 60 outside the vehicle are imaged so as to be included in one image.

The above description of the imageable region using the image pickup apparatus 20a is appropriately applied to other imaging apparatus 20b to 20d.

Next, the image-capable area, the image-imaging recommended area, and the image-imaging non-recommended area will be further described below.

FIG. 6 is a diagram for explaining an imaging enable region, an imaging recommended region, and an imaging non-recommended region.

As described above, the imageable region is a position on the path when the optical axis direction of the image pickup devices 20a to 20d represented by the map coordinate system intersects the position of the object to be imaged.

In the example shown in FIG. 6, the image pickup determination unit 11d determines that the image pickup device 20a mounted on the vehicle 50 has an image pickup possible region R in a predetermined range on the path 70. The image-capable area R is a region including the image-recommended area R2 and the non-imaging areas R1 and R3, which will be described later. As shown in FIG. 6, when the path is curved so as to rotate with the image pickup object 60 inside, the imageable region R may be provided in a predetermined range on the path 70.

The imaging determination unit 11d may record an image using the imaging device 20a in the imageable region R, or may capture images at predetermined intervals.

The imageable region R is usually a portion of a path 70 overlooking the image pickup object 60 from a position where the vehicle 50 travels.

Further, in the example shown in FIG. 6, the non-recommended imaging regions R1 and R3 in which it is not recommended to image the image pickup object 60 outside the vehicle are set on the path 70 on which the vehicle 50 on which the in-vehicle device 10 is mounted travels. ing.

In the imaging non-recommended region R1, an obstacle 80 such as a fence is located between the path 70 and the imaging object 60. Therefore, when the imaging device 20a images the image from inside the vehicle, the passenger and the imaging object 60 Since the obstacle 80 is captured in between, there is a possibility that the imaging target 60 outside the vehicle cannot be imaged.

Further, in the imaging non-recommended region R3, an obstacle 81 such as a building is located between the path 70 and the imaging object 60. Therefore, when the imaging device 20a images the image from inside the vehicle, the passenger and the imaging object 60 Since the obstacle 81 is captured between the vehicle and the vehicle, there is a possibility that the imaging target 60 outside the vehicle cannot be imaged.

On the other hand, between the non-recommended imaging region R1 and the non-recommended imaging region R3, a recommended imaging region R2 is set, which is recommended to image the imaging target 60 outside the vehicle.

The recommended imaging region R2 is, in particular, a portion on the path 70 where the imaging object 60 outside the vehicle is expected to be beautifully imaged. The recommended imaging region R2 is a so-called viewpoint.

The recommended imaging region information including the position information on the path 70 of the recommended imaging region R2 described above is registered in the recommended imaging region table 12e of the storage unit 12 of the vehicle-mounted device 10. In the recommended imaging region Table 12e, the identification information for identifying the recommended imaging region R2 and the position information on the path 70 of the recommended imaging region R2 are registered in association with each other.

Further, the non-recommended imaging region information including the position information on the path 70 of the non-recommended imaging regions R1 and R3 described above is registered in the non-recommended imaging region table 12f of the storage unit 12 of the in-vehicle device 10. In the imaging non-recommended region Table 12f, the identification information for identifying the imaging non-recommended regions R1 and R3 and the position information on the path 70 of the imaging non-recommended region R1 and R3 are registered in association with each other.

In the example shown in FIG. 6, the imageable region R has a region R4 that is neither a recommended imaging region nor a non-recommended imaging region between the non-recommended imaging region R3 and the recommended imaging region R2. In the region R4, the optical axis direction A1 of the image pickup apparatus 20a intersects the position of the image pickup object.

In the example shown in FIG. 6, the image pickup possible area, the image pickup recommended area, and the image pickup non-recommended area determined to be able to be imaged with respect to the image pickup apparatus 20a were described. The above description of FIG. 6 is appropriately applied to other imaging devices 20b to 20d.

Next, the server 40 will be described in detail below.

The server 40 has a processing unit 41, a storage unit 42, and a communication unit 43.

The processing unit 41 has one or more processors and peripheral circuits. The processing unit 41 controls each hardware component of the server 40 and performs various processes according to a predetermined program stored in the storage unit 42 in advance, and the storage unit 41 temporarily stores the data generated during the processing. Use 42. The number of servers does not have to be one. The operation of the server may be realized by a plurality of servers connected so as to be communicable via the network N in cooperation with each other.

The storage unit 42 may have a semiconductor memory such as a random access memory (RAM) or a read-only memory (ROM), or a non-volatile memory such as a magnetic disk or a flash memory. Further, the storage unit 42 may use a disk drive (not shown) as a drive capable of reading a non-temporary storage medium for storing a predetermined program.

FIG. 7 is a diagram illustrating a storage unit of the server.

The storage unit 42 stores the server imaging target object table 42a, the server imaging recommended area table 42b, and the server imaging non-recommended area table 42c.

In the server imaging object table 42a, the name of the imaging object and the position information of the imaging object are registered in association with the imaging object identification information for identifying the imaging object. As an imaging target, for example, a popular object as an imaging target can be registered on the Web. The information registered in the server imaging object table 42a is also referred to as imaging object information.

In the server imaging recommended area table 42b, the identification information for identifying the imaging recommended area and the position information on the path of the imaging recommended area are registered in association with each other. The recommended imaging area can be registered on the web as a portion on the path that is reputed to be suitable for imaging an object to be imaged.

In the server imaging non-recommended region table 42c, the identification information for identifying the imaging non-recommended region and the position information on the path of the imaging non-recommended region are registered in association with each other. The non-recommended imaging region can be registered on the web as a portion of the path that is unfortunately not suitable for imaging an object to be imaged.

The communication unit 43 transmits / receives information to / from the in-vehicle device 10 via the network N. The processing unit 41 performs various processes based on the information received by the communication unit 43. In addition, the processing unit 41 transmits the results of performing various processing using the communication unit 43. The communication unit 43 may have a communication circuit and a communication line or an antenna for transmitting and receiving.

Next, in the system 1, the in-vehicle device 10 determines the optical axis directions of the image pickup devices 20a to 20d and receives the image pickup object information from the server 40, etc., with reference to the sequence diagram shown in FIG. This will be described below.

First, in step S801, the passenger of the vehicle operates the optical axis adjusting units 22 of the imaging devices 20a to 20d to set the optical axis direction of the imaging unit 21. Specifically, the passenger of the vehicle adjusts the optical axis while checking the image of the display unit 13 so that the imaging unit 21 of each imaging device 20a to 20d can image the passenger and the scenery outside the vehicle together. The optical axis direction of the imaging unit 21 and the enlargement ratio of the image are adjusted by using the unit 22 or the operation unit 14. By adjusting the optical axis direction of the image pickup unit 21 and the enlargement ratio of the image, the proportion of the passenger in the field of view of the imaging unit 21 and the proportion of the outside of the vehicle are changed.

The optical axis direction determining unit 11a of the processing unit 11 of the in-vehicle device 10 determines the optical axis direction of each imaging unit 21 based on the movement amount information in the optical axis direction of the imaging unit 21 received from the imaging devices 20a to 20d. Determined for the vehicle coordinate system fixed to.

Next, in step S803, the image pickup object setting unit 11b of the processing unit 11 of the in-vehicle device 10 transmits the image pickup object information request requesting the image pickup object information to the server 40 via the network N. Further, the imaging target setting unit 11b transmits an imaging recommended region information request requesting imaging recommended region information and an imaging non-recommended region information request requesting imaging non-recommended region information to the server 40 via the network N. ..

Next, in step S805, the server 40 receives the image pickup object information request, the image pickup recommended area information request, and the image pickup non-recommended area information request.

Next, in step S807, the processing unit 41 of the server 40 reads the imaged object information by referring to the server imaged object table 41a stored in the storage unit 42, and reads the read imaged object information to the network. It is transmitted to the in-vehicle device 10 via N. Further, the processing unit 41 of the server 40 reads out the recommended imaging area information by referring to the server imaging recommended area table 42b stored in the storage unit 42, and transmits the read recommended imaging area information to the vehicle via the network N. It is transmitted to the device 10. Further, the processing unit 41 of the server 40 reads out the imaging non-recommended area information by referring to the server imaging non-recommended area table 42c stored in the storage unit 42, and uses the network N for the read imaging non-recommended area information. It is transmitted to the in-vehicle device 10 via the vehicle.

Next, in step S809, the image pickup object setting unit 11b receives the image pickup object information, the image pickup recommended area information, and the image pickup non-recommended area information.

Next, in step S811, the image pickup object setting unit 11b registers the received image pickup object information in the image pickup object table 12d of the storage unit 12. Further, the image pickup object setting unit 11b registers the received image pickup recommended area information in the image pickup recommended area table 12e of the storage unit 12. Further, the shadow object setting unit 11b registers the received image pickup non-recommended area information in the image pickup non-recommended area table 12f of the storage unit 12.

The passenger of the vehicle may operate the operation unit 14 of the in-vehicle device 10 to register the image-imaging object information of the image-imaging object to be imaged in the image-imaging object table 12d of the storage unit 12. Similarly, the passenger of the vehicle may register the image pickup recommended area information that he / she has found in the image pickup recommended area table 12e of the storage unit 12. Further, the passenger of the vehicle may register the image pickup non-recommended area information that he / she has found in the image pickup non-recommended area table 12f of the storage unit 12. As described above, the passenger of the vehicle may appropriately register the image pickup target information and the like in the in-vehicle device 10 in addition to the information stored in the server 40. Further, the imaging object setting unit 11b transmits information registered in addition to the imaging object table 12d, the imaging non-recommended area table 12f, or the imaging non-recommended area table 12f to the server 40, and sends the information to the storage unit 42. It may be registered in each stored table.

Next, in step S813, the imaging determination unit 11d of the processing unit 11 of the in-vehicle device 10 displays the three imaging regions selected by the passengers of the vehicle on the display unit 13.

FIG. 9 is a diagram showing a screen for selecting an imaging region.

When "1. Imaging in the imageable area" is selected, the image pickup determination unit 11d uses the image pickup devices 20a to 20d when the vehicle is located in the imageable area on the route to image the passenger and the outside of the vehicle. Take an image of an object. In the example shown in FIG. 6, while the vehicle is traveling in the imageable region R, the image pickup determination unit 11d uses the image pickup devices 20a to 20d to image the passenger and the image pickup object outside the vehicle. In this case, the imaging determination unit 11d may capture an image using the imaging devices 20a to 20d.

When "2. Imaging in the recommended imaging region" is selected, the imaging determination unit 11d sets the imaging devices 20a to 20d when the vehicle is located in the imageable region and the recommended imaging region on the route. It is used to image passengers and objects to be imaged outside the vehicle. As a result, even in the region on the path where the passenger and the object to be imaged outside the vehicle can be imaged, the object to be imaged can be imaged when it is expected that the object to be imaged can be imaged particularly wonderfully. In the example shown in FIG. 6, while the vehicle is traveling in the recommended imaging region R2, the imaging determination unit 11d uses the imaging devices 20a to 20d to image the passenger and the object to be imaged outside the vehicle.

When "3. Do not image in the non-recommended area for imaging" is selected, the image pickup determination unit 11d captures the image when the vehicle is located in the non-recommended area for imaging even if the vehicle is located in the non-recommended area for imaging. The devices 20a to 20d are not used to image the passenger and the object to be imaged outside the vehicle. As a result, even in the region on the path where the passenger and the object to be imaged outside the vehicle can be imaged, if it is expected that the object to be imaged cannot be imaged well, the image can be prevented. In the example shown in FIG. 6, even if the vehicle is traveling in the imageable region R, when the vehicle is located in the imaging non-recommended regions R1 and R3, the imaging determination unit 11d uses the imaging devices 20a to 20d. , Do not image the passengers and objects to be imaged outside the vehicle.

The image pickup determination unit 11d inputs an imaging region selected by the passenger of the vehicle using the operation unit 14, and uses the imaging devices 20a to 20d to take an image in the input imaging region while the vehicle is traveling.

Next, in step S815, the image pickup determination unit 11d of the processing unit 11 of the in-vehicle device 10 displays the two image pickup modes selected by the passengers of the vehicle on the display unit 13.

When "1. Imaging in synchronization with the first imaging device" is selected, the imaging determination unit 11d determines that the imaging device 20a that images the passenger seated in the driver's seat can be imaged. Occasionally, the image pickup device 20a is made to take an image, and other image pickup devices 20b to 20d are also made to take an image. As a result, the imaging device 20a images the passenger seated in the driver's seat and the object to be imaged outside the vehicle, and in synchronization with this imaging, captures another image of the passenger seated in another seat. Imaging can be performed using the devices 20b to 20d. This makes it possible to capture, for example, the facial expressions of each passenger in the magnificent scenery outside the vehicle. The first imaging device may be an imaging device other than the imaging device 20a.

When "2. Each imaging device independently images" is selected, the imaging determination unit 11d determines that the imaging devices 20a to 20d that image each passenger seated in the vehicle can be imaged. At that time, each imaging device 20a is made to take an image. Since the optical axis directions A1 to A4 of the imaging devices 20a to 20d are usually different, the timing at which it is determined that imaging is possible is also different. Therefore, the image pickup determination unit 11d can determine whether or not each image pickup device 20a to 20d can take an image, and when the image can be taken, can take an image of the passenger and the object to be imaged outside the vehicle, respectively.

The image pickup determination unit 11d inputs the image pickup mode selected by the passenger of the vehicle using the operation unit 14, and in the input image pickup mode, images are taken by using the image pickup devices 20a to 20d while the vehicle is traveling.

The above is the description of the process of the in-vehicle device 10 determining the optical axis direction of the image pickup devices 20a to 20d and receiving the image pickup object information from the server 40 in the system 1.

Next, the operation of the in-vehicle device 10 to image the image-imaging object while traveling on the route to the destination will be described below with reference to the flowcharts shown in FIGS. 11 and 12.

First, in step S1101, the navigation unit 11c of the processing unit 11 of the in-vehicle device 10 inputs a destination from the passenger of the vehicle and derives a route from the current position of the vehicle to the destination.

Next, in step S1103, the image pickup determination unit 11d is scheduled to travel on the route from the current vehicle position to the destination with reference to the map data 12b and the image pickup target table 12d stored in the storage unit 12. Search for an imaging object located within a predetermined distance from the route. Then, based on the selected imaging mode (see FIG. 10), the imaging determination unit 11d is on a path in which the optical axis direction of the imaging unit 21 intersects the position of the imaging target with respect to the imaging devices 20a to 20d. The position of is obtained as an imageable area. When an imaging mode for imaging in synchronization with the first imaging device (imaging apparatus 20a) is selected, the imaging determination unit 11d obtains an imaging possible region from the imaging device 20a. On the other hand, when the imaging mode in which each imaging device independently captures images is selected, the imaging determination unit 11d obtains an imaging capable region for each imaging device 20a to 20d. The reason for searching for an imaging object located within a predetermined distance from the planned travel route is that it is usually considered that an imaging object located too far from the route cannot be imaged. The predetermined distance is a distance in a direction orthogonal to the direction in which the path extends. The predetermined distance can be, for example, 1 km to 200 km. Depending on the weather, it may not be possible to take an image even if the object to be imaged is located within a predetermined distance. However, in the present specification, the imageable area is the object to be imaged by the passenger and the outside of the vehicle unless the weather is bad. It means a position on the path where an object can be imaged. Therefore, in the present specification, when the imaging devices 20a to 20d image the image-catching object of the passenger and the outside of the vehicle in the image-capable region, the image-imaging object actually takes an image depending on the weather or an unexpected condition. It is also included when it is not done. Unexpected conditions include, for example, the construction of a new building between the path and the object to be imaged.

When there is no imaging object located within a predetermined distance from the planned travel route, the navigation unit 11c continues to guide the route to the destination. Here, assuming that at least one imaging object has been searched, the following description will be given.

Then, the imaging determination unit 11d refers to the imaging recommended region table 12e or the imaging non-recommended region table 12f stored in the storage unit 12 based on the selected imaging region (see FIG. 9), and obtains the acquired imaging. The final imageable area is determined by excluding the area where imaging is not performed from the possible area.

The image pickup determination unit 11d may superimpose the name of the image pickup target on the map image and display it on the display unit 13 together with the route obtained by the navigation unit 11c. As a result, the passenger of the vehicle can grasp the position on the route where the image can be taken.

Next, in step S1105, the navigation unit 11c starts running the vehicle and also starts executing the navigation function to provide route guidance.

Next, in step S1107, the imaging determination unit 11d determines whether or not the predicted time to reach the imageable region has reached a predetermined time. The predetermined time is preferably a time during which the passenger who has been notified of the advance notice of the imaging can prepare for the imaging. For example, the predetermined time can be 10 seconds. Here, the predetermined time may have a certain range. For example, if the predicted time is between 9 seconds and 11 seconds as a predetermined time, it may be determined that the predicted time has reached the predetermined time. When there are a plurality of imageable areas, the predicted time means the time required to reach the imageable area closest to the current position of the vehicle in the traveling direction of the vehicle.

The imaging determination unit 11d obtains an estimated time to reach the imageable region based on the distance from the current position of the vehicle to the imageable region and the speed of the current vehicle. For example, when the speed of the vehicle is 50 km / h, the predicted time is 10 seconds at a position about 139 m before the imageable region. When the speed of the vehicle is 100 km / h, the predicted time is 10 seconds at a position about 278 m before the imageable region.

When the predicted time to reach the imageable area reaches a predetermined time (Yes in step S1107), the image pickup determination unit 11d indicates that the vehicle is approaching the imageable area. 13 or the acoustic output unit 15 is used to notify the passenger (step S1109).

FIG. 13 is a diagram showing a screen for notifying the imaging.

For example, as shown in FIG. 13, the image pickup determination unit 11d may display on the display unit 13 that "the passenger and the object to be imaged will be imaged after 10 seconds".

Further, the image pickup determination unit 11d counts down the time from the time when the vehicle notifies that the vehicle is approaching the image capture area to the time when the vehicle reaches the image capture area, and uses the display unit 13 or the sound output unit 15 to board the vehicle. May be notified. As a result, the passenger who is seated in a place other than the driver's seat can pose toward the image pickup unit 21 to prepare for the image pickup.

On the other hand, if the predicted time to reach the imageable region is not a predetermined time (No in step S1107), the process returns to before step S1105.

Next, in step S1111, the navigation unit 11c executes the navigation function and continues the route guidance.

Next, in step S1113, the image pickup determination unit 11d determines whether or not the vehicle has reached the imageable region. As described above, in the imageable region, in the map coordinate system, the optical axis direction of the image pickup unit 21 intersects with the image pickup object, and the passenger and the image pickup object outside the vehicle are used by using the image pickup devices 20a to 20b. Can be imaged so as to be included in one image.

When it is determined that the vehicle has reached the image-capable area (Yes in step S1113), the image-imaging determination unit 11d is boarded using the image-imaging devices 20a to 20d determined to be image-capable based on the image pickup mode. The person and the object to be imaged outside the vehicle are imaged (step S1201). FIG. 14 shows an image of a passenger sitting in the driver's seat and an object to be imaged outside the vehicle using the image pickup device 20a. The image pickup determination unit 11d stores the captured image as image data 12c in the storage unit 12, and the image pickup determination unit 11d may transmit the captured image to the server 40 via the network N. The server 40 may store the received image in the storage unit 42.

On the other hand, when it is determined that the vehicle has not reached the imageable region (No in step S1113), the process returns to the front of step S1111.

Next, in step S1203, the navigation unit 11c executes the navigation function and continues the route guidance.

Next, in step S1205, the navigation unit 11c determines whether or not the vehicle has arrived at the destination. When the vehicle arrives at the destination (yes in step S1205), the in-vehicle device 10 ends the operation of imaging the object to be imaged. On the other hand, if the vehicle has not arrived at the destination (No in step S1205), the vehicle returns to the front of step S1105.

According to the in-vehicle device 10 of the system 1 of the present embodiment described above, it is possible to take an image so that the passenger on the vehicle and the object to be imaged outside the vehicle are included in one image while the vehicle is running.

For example, conventionally, when the driver is the only passenger in the vehicle, the driver stops the vehicle to take an image of an object to be imaged outside the vehicle. It was. On the other hand, according to the in-vehicle device 10 of the present embodiment, it is possible to take an image of a passenger sitting in the driver's seat and an object to be imaged outside the vehicle without stopping the moving vehicle.

Further, conventionally, in a moving vehicle, when a passenger takes an image of an object to be imaged outside the vehicle, the image may be deviated from the most suitable timing for the image. On the other hand, according to the in-vehicle device 10 of the present embodiment, since the image pickup possible area on the path is obtained and the image pickup is performed, the image pickup can be performed at the most suitable timing for the image pickup.

In the present invention, the in-vehicle system, the in-vehicle device, the program, and the method of the above-described embodiment can be appropriately changed as long as they do not deviate from the gist of the present invention.

For example, the in-vehicle device described above may be a high-performance mobile terminal. The above-mentioned in-vehicle device can also be realized by using a high-performance mobile terminal having a navigation function and an imaging function.

FIG. 15 is a diagram showing a high-performance mobile terminal as another embodiment of the in-vehicle device.

The high-performance mobile terminal 90 has an image pickup unit 91, and a touch panel 92 as a display unit and an operation unit. Further, the high-performance mobile terminal 90 has a processing unit, a storage unit, an acoustic output unit, a communication unit, and a GPS information receiving unit (not shown). The high-performance mobile terminal 90 may be connected to the CAN to receive the speed of the vehicle, or may obtain the speed of the vehicle based on the change in the current position.

The high-performance mobile terminal 90 is fixed in the vehicle interior by using the mounter 100.

The mounter 100 has a movable portion 101 for fixing the high-performance mobile terminal 90, and a fixed portion 102 for rotatably fixing the movable portion 101 in the vehicle interior.

The movable portion 101 has four locking portions 101a to 101d, and the high-performance mobile terminal 90 is fixed to the movable portion 101 by using these locking portions 101a to 101d.

The fixing portion 102 is fixed in the vehicle interior based on a predetermined positional relationship with respect to the vehicle coordinate system. For example, the fixing portion 102 is fixed at a position on the left end of the dashboard. As a result, the optical axis direction of the image pickup unit 91 can be set so as to image the passenger seated in the driver's seat and the object to be imaged outside the vehicle by using the image pickup unit 91 of the high-performance mobile terminal 90.

The movable portion 101 is rotatable with respect to the fixed portion 102 as shown in the direction of the arrow in FIG. As a result, the direction of the optical axis of the imaging unit 91 can be set so as to change the direction of the high-performance mobile terminal 90 fixed to the movable unit 101 and image the outside of the vehicle including the passengers inside the vehicle.

On the fixed portion 102, an index 102a indicating the positional relationship of the movable portion 101 with respect to the fixed portion 102 is displayed.

An optical axis direction index 101e is displayed at the lower part of the movable part 101, and by reading the optical axis direction index 101e at a position corresponding to the index 102a, the positional relationship of the movable part 101 with respect to the fixed part 102 can be obtained. Has been done.

The passenger of the vehicle sets the optical axis direction and the magnification of the imaging unit 91 while looking at the image on the touch panel 92, then reads the optical axis direction index 101e at the position corresponding to the index 102a, and reads the optical axis direction index 101e. Is input to the high-performance mobile terminal 90.

The high-performance mobile terminal 90 sets the optical axis direction of the imaging unit 91 based on the input optical axis direction index 101e and the predetermined positional relationship of the fixed unit 102 fixed to the vehicle interior with respect to the vehicle coordinate system. Determined for the vehicle coordinate system fixed to.

The high-performance mobile terminal 90 can operate in the same manner as the in-vehicle device 10 of the above-described embodiment, and based on the positional relationship between the optical axis direction of the image pickup unit 91 and the image pickup object outside the vehicle, the passenger and the image pickup are performed. It is determined whether the object can be imaged, and if the object can be imaged, the passenger and the object to be imaged are imaged by using the imaging unit 91.

Further, in the above-described embodiment, the processing unit 11 has determined that the passenger and the imaging object can be imaged when the optical axis direction of the imaging unit 21 intersects the position of the imaging object. When the image pickup object is included in the field of view of the image pickup unit 21, it may be determined that the passenger and the image pickup object can be imaged.

1 System 10 In-vehicle device 11 Processing unit 11a Optical axis direction determination unit 11b Imaging object setting unit 11c Navigation unit 11d Imaging judgment unit 12 Storage unit 12a Program 12b Map data 12c Image data 12d Imaging object table 12e Imaging recommended area table 12f Imaging Non-recommended area Table 13 Display (output)
14 Operation unit 15 Acoustic output unit 16 Communication unit (output unit)
20a to 20d Imaging device 21 Imaging unit 22 Optical axis adjustment unit 30 GPS information receiving unit 40 Server 41 Processing unit 42 Storage unit 43 Communication unit 50 Vehicles 51a to 51d Passenger 60 Imaging object 70 Route 80 Obstacle 81 Obstacle 90 Terminal 91 Imaging unit 100 mounter 101 movable unit 101a-101d locking unit 101e optical axis direction index 102 fixed unit 102a index N network P image

Claims (8)

An image pickup unit that can set the optical axis direction so as to image the outside of the vehicle including the passengers inside the vehicle
When the position of the imaging object outside the vehicle is acquired and the optical axis direction of the imaging unit intersects the position of the imaging object based on the positional relationship between the optical axis direction of the imaging unit and the imaging object outside the vehicle. It is determined whether the passenger and the object to be imaged can be imaged so that the passenger and the object to be imaged are included in the scenery outside the vehicle, and if the image can be taken, the image including the passenger and the object to be imaged is imaged on the imaging unit. The processing unit to be made to
An output unit that outputs the captured image and
In-vehicle system equipped with. The processing unit seeks an imaging position capable of imaging the occupant and the object to be imaged on the route on which the vehicle on which the imaging unit is mounted travels, and before the vehicle reaches the imaging position, The in-vehicle system according to claim 1, wherein the passenger is notified that the passenger and the object to be imaged are imaged. A recommended imaging region is set on the route on which the vehicle on which the imaging unit is mounted travels, and it is recommended to image an object to be imaged outside the vehicle.
The processing unit determines that the occupant and the object to be imaged can be imaged, and causes the image pickup unit to image the occupant and the object to be imaged when the vehicle is located in the recommended imaging area. The in-vehicle system according to 1 or 2. An imaging non-recommended area is set in the route on which the vehicle on which the imaging unit is mounted travels, and it is not recommended to image an imaging object outside the vehicle.
Even if the processing unit determines that the occupant and the object to be imaged can be imaged, the imaging unit does not allow the occupant and the object to be imaged to be imaged when the vehicle is located in the non-recommended area for imaging. The in-vehicle system according to any one of Items 1 to 3. The image pickup unit has a first image pickup unit having a first optical axis direction and a second image pickup unit having a second optical axis direction.
The processing unit determines whether the passenger and the imaging object can be imaged based on the positional relationship between the first optical axis direction of the first imaging unit and the imaging object outside the vehicle, and if it is possible to image the image. The in-vehicle system according to any one of claims 1 to 4 , wherein the passenger and the object to be imaged are imaged by the first imaging unit and also the second imaging unit. Based on the positional relationship between the optical axis direction of the imaging unit and the imaging object outside the vehicle, which can acquire the position of the imaging object outside the vehicle and set the optical axis direction so as to image the outside of the vehicle including the passengers inside the vehicle. When the optical axis direction of the imaging unit intersects the position of the imaging object, it is determined whether the passenger and the imaging object can be imaged so that the passenger and the imaging object are included in the outside scenery of the vehicle, and imaging is performed. A processing unit that causes the imaging unit to image an image including the passenger and the object to be imaged when possible.
In-vehicle device including. Acquiring the position of the object to be imaged outside the vehicle
Based on the positional relationship between the optical axis direction of the imaging unit whose optical axis direction can be set so as to image the outside of the vehicle including the passengers inside the vehicle and the imaging object outside the vehicle, the optical axis direction of the imaging unit is determined. Judging whether it is possible to capture an image including the passenger and the object to be imaged so that the passenger and the object to be photographed are included in the scenery outside the vehicle when the position of the object to be imaged intersects.
A program that causes a processor to capture an image of a passenger and an object to be imaged using the imaging unit when imaging is possible. Acquiring the position of the object to be imaged outside the vehicle
Based on the positional relationship between the optical axis direction of the imaging unit whose optical axis direction can be set so as to image the outside of the vehicle including the passengers inside the vehicle and the imaging object outside the vehicle, the optical axis direction of the imaging unit is determined. Judging whether it is possible to capture an image including the passenger and the object to be imaged so that the passenger and the object to be photographed are included in the scenery outside the vehicle when the position of the object to be imaged intersects.
When it is possible to take an image, the image pickup unit is used to take an image including the passenger and the object to be imaged.
How to prepare.

Images