JP4051990B2 - Safe driving support device and image processing device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
  The present invention relates to a safe driving support device that detects and uses objects such as signs, signs, and traffic lights from images taken by an in-vehicle camera so that the driver can concentrate on driving.And image processing apparatusAbout.
[0002]
[Prior art]
  Conventionally, there is a technique described in Japanese Patent Laid-Open No. 6-76065 as a technique for detecting and displaying a sign or the like from an in-vehicle camera image. FIG. 13 shows a configuration diagram of the prior art. The image data input by the image input unit 102 is stored in the image buffer 201. The sample area designating unit 1301 is a component for registering a sign or the like that the user wants to recognize. The feature point extraction process 1302 is a process for extracting a part of the feature of the image data. If the detection target is a sign, the region extraction process 1303 extracts a sign candidate from the feature, and the detection target is a lane. If there is, a feature point interpolation process 1304 extracts a lane from the feature. When the detection target is a sign, the image data comparison processing 1305 further determines whether the sign data is designated by comparing the characteristics of the input image data with the sample data registered in the sample data storage memory 1306. The display means 106 displays the input image and the recognition result. FIG. 14 is a diagram showing a display method. A recognition result display portion 1402 is provided on the display screen 1401 of the display means 106, and an image such as a recognized road sign is displayed on the recognition result display portion 1402.
[0003]
  Conventionally, when a car crossing is reached with the volume of the car audio raised, the driver has lowered the volume of the car audio himself to hear whether the crossing is sounding.
[0004]
  Further, Japanese Patent Application Laid-Open No. 9-185703 discloses a technique for detecting and notifying height restriction, width restriction, weight restriction signs, and the like from road signs among road signs. In the prior art, these road signs are detected from in-vehicle camera images, and the driver is notified of the presence of the road signs.
[0005]
[Problems to be solved by the invention]
  However, in the prior art described in Japanese Patent Laid-Open No. 6-76065, a recognized detection target (a label or the like) is displayed as an image on the display screen of the display unit 106 as it is, and an image around the detection target is displayed. Therefore, there is a possibility that auxiliary signs and auxiliary signals under signs and traffic lights may not be displayed. The driver had to turn his eyes to the display means every time the detection target was detected, and could not concentrate on safe driving.
[0006]
  An object of the present invention is to provide a system that can support the driver's safe driving by solving the above-described problems and presenting and using truly necessary information without disturbing the driver's driving.
[0007]
[Means for Solving the Problems]
  The present invention is a detection target from an image.Is a traffic lightA safe driving support apparatus that includes an image detection means for detecting the presence of the detection target and presents the presence of the detection target to a driver using a display means.Enlarge the area below the traffic light that isAn area obtained by enlarging the detection area to be detected based on the area enlargement information using area enlargement informationimageOf the enlarged region portion ofPictureAn image is presented on the display means.
[0008]
DETAILED DESCRIPTION OF THE INVENTION
    One embodiment of the present invention is a safe driving support apparatus that includes an image detection unit that detects a detection target from an image and presents the presence of the detection target to a driver using a display unit. Using the area enlargement information prepared in advance, an area image obtained by enlarging the detection area to be detected based on the area enlargement information is obtained, and the image of the enlarged area portion is presented on the display means.
[0009]
Preferably, the detection target is a traffic light, and the area enlargement information includes an amount of enlargement of the area below the traffic light, obtains an area image obtained by enlarging the detection area based on the area enlargement information, and the enlargement An image of the area portion that has been displayed on the display means,Since not only traffic lights but also auxiliary signals (signals such as “→”) can be displayed at the same time, it is possible to concentrate on safe driving without having to stare at actual small auxiliary signals.
[0010]
  Furthermore, it is preferable that an image of the enlarged area portion is presented to the display means only when the traffic light is a red signal, and only when the auxiliary signal (a signal such as “→”) is lit up. SignalSo you do n’t have to stare at the actual small auxiliary signal,In addition, it is not necessary to keep an eye on the display means every time it passes through a traffic light other than a red signal.Focus on safe driving.
[0011]
One embodiment of the present inventionFrom an image input means for inputting an image, an image detection means for detecting a road traffic sign or traffic light as a detection target from an image input from the image input means, and a storage means for storing presentation pattern information related to the road traffic sign or traffic light in advance Presentation means for selecting a presentation content from the acquired presentation pattern information and a detection target detected by the image detection means, wherein the presentation pattern information includes area expansion information, and the detection target is based on the area expansion information Is an image processing apparatus that presents an image of a region portion in which the detection region of the image is enlarged on the display means.
[0012]
Preferably, the presentation pattern information includes information on an enlargement direction and an enlargement ratio. is there.
[0013]
  Preferably, the presentation pattern information includes information related to the presentation time..
[0014]
  Although not directly related to the present invention, there are the following two points as other problems with respect to the prior art. First of all, when approaching a railroad crossing with the volume of the car audio turned up, the driver operates the volume of the car audio to pay attention to whether the railroad crossing is ringing, which may interfere with safe driving. It was.
[0015]
  Other reference embodiments for this problemA safe driving support device for controlling the operation of the in-vehicle device when the detection target is detected by the image detection means, the detection target being a level crossing caution. The vehicle-mounted device is a car audio, and is configured to reduce the volume of the car audio when a traffic sign of a crossing caution that is the detection target is detected. As a result, the driver can hear the sound of the railroad crossing without operating the volume of the car audio when the vehicle approaches the railroad crossing with the volume of the car audio raised, and can concentrate on safe driving.
[0016]
  As a second other problem, in the prior art described in Japanese Patent Laid-Open No. 9-185703, a warning is always issued on a road having a height limit, a width limit and a weight limit, and the own vehicle exceeds the limit. I had to judge whether or not the driver himself could pass. Moreover, the driver did not remember the exact height, width and weight of the car he was on, and was driving based on the assumption that he would be able to pass while confirming whether he could pass. For this reason, there was a possibility that the driver could not concentrate on driving or, in some cases, misjudged and damaged the car.
[0017]
  Another reference embodiment for this problem is a safe driving support apparatus that includes an image detection unit that detects a detection target from an image and presents the presence of the detection target to a driver using a presentation unit. Vehicle state detecting means for detecting the state, wherein the vehicle state is at least one of height, width and weight of the vehicle, and the detection object is a traffic sign of height restriction, width restriction and weight restriction When at least one of the traffic signs is detected, if at least one of the height, width and weight of the vehicle in the vehicle state does not satisfy the limit of the detected traffic signs, the detection is made to the driver. It is configured to present that it does not meet the traffic sign restrictions imposed. This allows the driver to know in advance whether or not he / she can pass even if he / she does not remember the detailed height / width / weight of the car when he / she approaches a road with height / width / weight restrictions. A quick decision can be made.
[0018]
DETAILED DESCRIPTION OF THE INVENTION
  Hereinafter, embodiments of the present invention will be described with reference to FIGS.
[0019]
  (Embodiment 1)
  FIG. 1 shows a block diagram of a system that can be realized by the present invention. Image data is obtained from an image input means (vehicle camera) 102 arranged so as to photograph the front of the car 101 and stored in the storage means 103. Then, as will be described later, the CPU 104 reads out the image data from the storage means 103, detects a predetermined detection target from the image data, processes the detection result into a form that can be presented to the driver, the speaker 105, the display The output to the means 106 or other vehicle-mounted device 107 is controlled. The input means 108 includes a keyboard and a mouse, and can be used to edit a presentation pattern to be described later.
[0020]
  The operation of the system configured as described above will be described below. The image input means 102 is installed so as to photograph the front of the car 101, and stores the obtained image in the storage means 103. It is assumed that the storage means 103 also stores objects to be detected in advance (for example, road traffic signs, traffic lights, etc.) and programs executed by the CPU 104.
[0021]
  The CPU 104 reads the program from the storage unit 103 and executes it. FIG. 2 shows a configuration diagram of processing performed by the CPU 104. The image data input from the image input unit 102 is stored in the image buffer 201 in the storage unit 103. The image detection unit 202 detects a predetermined detection target from the image data stored in the image buffer 201 and outputs the detection result.
[0022]
  As an example, the image detection means 202 can use “high-speed object search using local color information—active search method” (Science Theory VOL.J81-DII, No.9, pp.2035-2042). . The detection result output from the image detection means 202 is composed of the name of the detection target and the detected position in the image data as shown below. The detected position is represented by a circumscribed rectangle surrounding the detection target, and the circumscribed rectangle is represented by xy coordinates on the upper left and lower right.
[0023]
  (Name, upper left x coordinate, upper left y coordinate, lower right x coordinate, lower right y coordinate)
  FIG. 3 shows an example of the detection result. FIG. 3A shows an example in which there is a traffic light (named SIGNAL) in the image data. At this time, if the image detection unit 202 is set to detect a traffic light (SIGNAL), the detection result is a set of a detection target name SIGNAL and a detected position as follows.
[0024]
  (SIGNAL, sx0, sy0, ex0, ey0)
  As another example, there is a road sign with a speed limit of 50 km / h (named LS50) as shown in FIG. 3B, and the image detecting means 202 is set to detect the road sign with a speed limit of 50 km / h. If so, the detection result is
  (LS50, sx1, sy1, ex1, ey1). When there are a plurality of detection targets, a set of sets of detection target names and detection positions is obtained.
[0025]
  Next, in FIG. 2, the detection result processing unit 203 selects a presentation pattern corresponding to the detection result by referring to a presentation pattern storage table 204 prepared in advance, and the presentation unit described in the presentation pattern The presentation content is output in the format specified in 205. The presentation unit 205 corresponds to the speaker 105, the display unit 106, and the in-vehicle device 107 in FIG.
[0026]
  The presentation pattern includes a target name detected as follows, presentation means for outputting the presentation contents, an output format, and presentation contents (a plurality of parameters).
[0027]
  (Subject name, presentation means, output format, parameter 1, parameter 2, ..., parameter N)
  Thus, the reason for making the presentation pattern common to all the presentation means is to allow the user to easily change the favorite presentation method for each detection target using the input means 108 such as a keyboard. For example, if it is not preferable to detect the traffic signal and display it on the display means, the presentation method can be freely changed even after the system is constructed, such as by using a speaker to notify the presence of the traffic signal by voice.
[0028]
  Here, the presentation means may be control of the speaker 105, the display means 106, the in-vehicle device 107, and the like, and is specified by using an identification name (in parentheses) for indicating the presentation means 205. An example of each presentation means is shown below.
[0029]
  As the speaker 105,
    Audio speaker (SP)
    Earphone (EPS)
    There are headphones (HPS). Although these speakers are of different types, they can output voice and warning sounds by the same operation, and include A / D conversion and amplifiers for reproducing sound data.
[0030]
  As the display means 106,
    Car navigation monitor (CNM)
    TV monitor (TVM)
    Dashboard monitor (DBM)
    Head-up display (HUD)
    There is a head mounted display (HMD). Although these types of display means are different, an image can be displayed at a specific position by the same operation, and a circuit for driving the display means is also included.
[0031]
  As the in-vehicle device 107,
    Car audio mute function (MUTE)
    Horn (HORN)
    There is a headlight (LIGHT) or the like, which can be turned on / off with a digital signal.
[0032]
  FIG. 4 shows an example of the output format to each presentation means. For example, in the case of a display means, there is only one output format (format 1). It is predetermined that the output format 1 (PAT1) of the display means requires four parameters including area expansion information (parameters 1 and 2).
[0033]
  The operation will be described with reference to a flowchart showing the operation of the detection result processing means of FIG. First, in step 501, it is determined whether or not the detection result to be processed remains. In step 502, the detection result is read, and the detected target name (named NAME) and coordinates are read. In step 503, a presentation pattern whose target name is NAME is acquired from the presentation pattern storage table 204. In step 504, data necessary for the presentation pattern is generated, and data generated in the format designated by the presentation means 205 designated by the presentation pattern is output. Steps 501 to 504 are performed for all detection results. If there are a plurality of presentation patterns to be applied to one detection result in step 503, step 504 is applied to all the presentation patterns. In step 504, the operation differs depending on the presentation pattern, and the operation will be described below using a typical example.
[0034]
  FIG. 6 shows one example of presentation using display means. In this example, a red signal (SIGNAL-R) is detected, and the detection result is
  (SIGNAL-R, sx0, sy0, ex0, ey0), and the pre-registered presentation pattern is
  (DBM, PAT1, PARAM1 = 3 (downward), PARAM2 = 2 (times), PARAM3 = (dsx0, dsy0, dex0, dey0), PARAM4 = 3 (seconds)). Parameter 1 is the direction of enlargement, and the direction and number are associated with each other in FIG. Parameter 2 is an enlargement ratio, and the direction of parameter 1 is multiplied by the number of parameters 2 (here, 2) multiplied by the height of the detected sign (ey0-sy0) as shown in FIG. The detection area is expanded (downward). Parameter 3 is the display area on the display means, and parameter 4 is the presentation time. In this case, the detection area ((sx0, sy0), (ex0, ey0)) of the detection target (SIGNAL-R) in the input image is doubled downward, and an image in the enlarged area is output. The upper left and lower right coordinates of the previous dashboard monitor 601 (DBM) are enlarged (or reduced) so as to fit the display area 602 where (dsx0, dsy0) and (dex0, dey0) are displayed. As a result, as shown in FIG. 6C, the display area 602 designated by PARAM3 of the dashboard monitor (DBM) 601 is a partial image of the area where the height of the detection area is doubled in the downward direction (PARAM1). The image is enlarged (or reduced) to 3 seconds and displayed. Rather than using the image recognition technology to detect auxiliary signals that are difficult to detect in this way, a relatively easy-to-detect signal is detected and the surrounding video is always cut out at the same time so that the auxiliary signal is easily perceived by the driver. I am trying to let you.
[0035]
  FIG. 7 shows another presentation example. In this example, it is assumed that the road sign “limit speed 50” (LS50) is detected, and the detection result is
  (LS50, sx1, sy1, ex1, ey1) and the pre-registered presentation pattern is
  (LS50, DBM, PAT1, PARAM1 = 3 (downward), PARAM2 = 1.5 (times), PARAM3 = display area, PARAM4 = 3 seconds). Parameter 1 is the direction of enlargement, and the direction and number are associated with each other in FIG. Parameter 2 is an enlargement ratio. As shown in FIG. 7A, the parameter 1 is multiplied by the number of parameter 2 (here, 1.5) multiplied by the height of the detected sign (ey1-sy1). Magnify in the direction (downward). Parameter 3 is the display area on the display means, and parameter 4 is the presentation time. In this case, the detection area ((sx1, sy1), (ex1, ey1)) of the detection target (LS50) in the input image is enlarged 1.5 times downward, and an image in the enlarged area is output. The upper left and lower right coordinates of the previous dashboard monitor 601 are enlarged (or reduced) so as to fit in a rectangular area where (dsx0, dsy0) and (dex0, dey0). As a result, as shown in FIG. 7B, the area expanded 1.5 times in the downward direction (PARAM1) is expanded (or reduced) to the display area 701 designated by PARAM3 of the dashboard monitor (DBM) 601. For 3 seconds.
[0036]
  The examples of FIGS. 6 and 7 show the effect of making it easier for the driver to perceive auxiliary signals and auxiliary signs by detecting traffic lights and road signs and simultaneously cutting out and enlarging the surrounding images. Although the auxiliary signals and signs accompanying traffic lights and road signs are very important information, they are difficult to perceive by the driver and difficult to detect by image detection means. It is.
[0037]
  Next, the case where a road condition is presented via an in-vehicle device will be described. In FIG. 4, there is one output format, and format 1 (PAT1) turns on (or turns off) the control signal of the specified in-vehicle device. Parameter 1 represents which in-vehicle device is controlled in a bit sequence, and the correspondence between the bit position and the control device is determined in advance. For example, the first bit (least significant bit) is a car stereo mute (MUTE), the second bit is a headlight (LIGHT), the third bit is a horn (HORN), and so on. Parameter 2 represents the blinking interval (presentation cycle) in seconds, and parameter 3 represents the presentation time.
[0038]
  The example of a presentation accompanying control of vehicle equipment is shown. In this example, it is assumed that the road sign “Cross-crossing caution” (a warning sign) is detected,
  (RAILROAD, 100, 120, 200, 225), and the pre-registered presentation pattern is
  (RAILROAD, AUDIO, PAT1, PARAM1 = MUTE, PARAM2 = 30 seconds, PARAM3 = 30 (seconds)). Parameter 1 indicates that the car audio mute (MUTE) is controlled, parameter 2 indicates that the mute ON / OFF cycle is 30 seconds, and parameter 3 indicates that the operation is continued for 30 seconds. Yes. As a result, the sound is muted for 15 seconds, and then the volume of the car audio is restored.
[0039]
  In this way, when the “crossing” sign is detected forward, the car audio is muted for 15 seconds so that the driver can hear the sound of the crossing without operating the car audio, so that safe driving can be performed. .
[0040]
  Moreover, another example of presentation accompanied by control of the in-vehicle device is shown. In this example, it is assumed that the road sign “warning alarm” is detected, and the detection result is
  (HORN, 100, 120, 200, 225), and the pre-registered presentation pattern is
  (HORN, HORN, PAT1, PARAM1 = 0.5 (seconds), PARAM2 = 1 (seconds)). HORN indicates that the horn is controlled, parameter 1 indicates that the horn ON / OFF cycle is 0.5 seconds, and parameter 3 indicates that the operation is continued for 1 second.
[0041]
  Thus, when the “warning alarm” sign is detected in the front, the 0.25 second horn is leveled for 1 second at 0.25 second intervals. As a result, the horn that lasts for 0.25 seconds is made twice, and the driver can automatically sound the horn when the steering wheel is busy due to a curve or the like without having to release his hand, and can concentrate on driving.
[0042]
  FIG. 8 shows a timing chart when the above-mentioned presentation pattern is used when this “warning narration” is detected. When the image input means inputs a moving image, the image data is usually sampled at a period of 1/30 seconds and input as a frame image. At time T1, the input of the first frame image is completed. At time T2, an “alarm alert” indicator is detected, and at the same time, horn control (horn ON) starts. ON / OFF is repeated at a period of 0.5 seconds from time T2, and this is continued for 1 second. As a result, a 0.25 second horn is sounded twice between time T2 and time T3.
[0043]
  Furthermore, presentation by voice used in the second embodiment to be described later will be described. In FIG. 4, there are two speaker output formats. Format 1 (PAT1) outputs the sound data specified by parameter 1 as many times as specified by parameter 2. In format 2 (PAT2), the text data specified by parameter 1 is converted into sound data by a speech synthesis means or the like, and the number of presentations specified by parameter 2 is output. This format 2 has an advantage that the driver can edit the presentation pattern by using the input means 108 such as a keyboard and can utter in a favorite word.
[0044]
  In the case of the presentation pattern presented to the speaker in the format 1, first, the sound data designated by the parameter 1 is read from the storage means 103. Then, the sound data is output for the number of times determined by parameter 2. When presenting to the speaker in the format 2, the text data specified by the parameter 1 (the content to be uttered) is converted into voice data by using the voice synthesis means, and the voice data is transmitted from the speaker by the number of times specified by the parameter 2. Output. A method for realizing speech synthesis is described in “New Edition, Information Processing Handbook” (p.1329, edited by Information Processing Society of Japan). An example in which a speaker is used as a presentation unit will be described in a second embodiment described later.
[0045]
  The time measurement can be realized by using a timer function of the CPU.
[0046]
  In addition, the vehicle-mounted device 107 in FIG. 1 may include a brake, an accelerator, a handle, a headlight, a high beam, and the like, and these devices can be controlled according to the detection result. For example, it is possible to turn on the brake if a “stop” sign is detected, or to turn off the high beam if “crosswalk” is detected.
[0047]
  Furthermore, as the presentation pattern, not only the presentation pattern is selected from the storage table 204, but also the presentation pattern adapted to the occasion may be created by the search result processing means 203.
[0048]
  As described above, it is possible to notify the road environment without causing confusion for the driver who concentrates on driving by changing the presenting means and presenting contents according to the detected contents.
[0049]
  (Embodiment 2)
  FIG. 9 shows a block diagram of a system that can be realized by the present invention. Image data is obtained from an image input means (vehicle camera) 102 arranged so as to photograph the front of the car 101 and stored in the storage means 103. Then, the CPU 104 reads out the image data from the storage means 103, detects a predetermined detection target from the image data, and in parallel, the vehicle state detection means 901 detects the vehicle state (for example, vehicle height, vehicle width, In addition to the vehicle weight, it detects whether the fuel is full, speed (including stopping), and whether the vehicle is moving backward. These detection results are processed into a form that can be presented to the driver, and output to the speaker 105 and the display device 106, or other vehicle-mounted devices 107 are controlled.
[0050]
  The operation of the system configured as described above will be described below. The image input means 102 is installed so as to photograph the front of the car 101, and stores the obtained image in the storage device 103. Assume that the storage means 103 stores in advance a detection target (for example, a road traffic sign, a traffic light, a convenience store, a gas station, etc.) and a program executed by the CPU 104.
[0051]
  The CPU 104 reads the program from the storage unit 103 and executes it. FIG. 10 shows a configuration diagram of processing performed by the CPU 104. The image data input from the image input unit 102 is stored in the image buffer 201 in the storage unit 103. The image detection unit 202 detects a predetermined detection target from the image data stored in the image buffer 201 and outputs the detection result. Since the image detection unit 202 and the data format of the detection result are the same as those in the first embodiment, description thereof will be omitted. The detection result processing unit 1001 outputs the detection contents to the presentation unit 205 in consideration of the detection result and the vehicle state described below.
[0052]
  The vehicle state detection means 901 includes various sensors (fuel tank / oil remaining amount, gear state, turn signal lighting state, steering angle, vehicle speed, etc.) and vehicle-specific data (vehicle height, vehicle width, You can know the state of the car from its own weight. Here, the state of the vehicle output from the vehicle state detection means 901 will be described. The types of vehicle conditions include the remaining amount of fuel, the remaining amount of various oils (engine oil, brake oil), the state of gears, the blinker lighting state, rudder angle, vehicle speed, vehicle height, vehicle width, weight, etc. is there. The remaining amount of fuel and the remaining amount of various oils are expressed by 1 bit each, and are set to “1” when the remaining amount is smaller than a predetermined amount, and “0” otherwise. The state of the gear is expressed by 1 bit indicating whether or not the vehicle is in the reverse state, “1” if it is a reverse gear, “0” otherwise. The blinker lighting is expressed by 2 bits, “01” if it is right, “10” if it is left, and “00” if neither is lighting. The rudder angle, vehicle speed, vehicle height, vehicle width, and own weight are expressed by quantizing the numerical values with 8 bits each. These are put together to express the state of the car with 46 bits. In the following, for convenience, the upper 2 bits are added to form 48 bits, which are expressed in hexadecimal (12 digits prefixed with “0x”). In the following description, equations for obtaining the vehicle speed, vehicle height, vehicle width, and own weight from the 48-bit data are defined as follows.
[0053]
  GETSPEED (x) = (x AND 0x0000FF000000) >> 24 can extract only the vehicle speed from the vehicle state x. “AND” and “>>” are logical operations, and “A >> B” indicates that A is shifted B bits to the right (left is zeroed). Similarly, to extract the vehicle height, vehicle width and weight from the vehicle state x,
  GETHEIGHT (x) = (x AND 0x000000FF0000) >> 16
  GETWIDTH (x) = (x AND 0x00000000FF00) >> 8
  GETWEIGHT (x) = (x AND 0x0000000000000FF). In addition, the rudder angle is an integer with a sign, and the front of the car is 0 degree, the right direction is positive and the left direction is negative. The formula for obtaining the rudder angle from the vehicle state x is
  GETDAKAKU (x) = INT ((x AND 0x00FF00000000) >> 32). INT (x) returns a signed integer value, where x is an integer expressed in two's complement.
[0054]
  Next, data (presentation pattern) stored in the presentation pattern storage table 1002 will be described. As in the first embodiment, the presentation method (presentation pattern) is determined in advance for each detection target, and held in the presentation pattern storage table 1002. The presentation pattern includes a detected target name, a condition relating to the state of the vehicle, a presentation means for outputting the presentation content, an output format, and a presentation content (a plurality of parameters) as shown below.
[0055]
  (Subject name, vehicle condition, presentation means, output format, parameter 1, parameter 2, ..., parameter N)
  Here, since the target name, presentation means, output format, and parameters 1 to N are the same as those in the first embodiment, description thereof will be omitted.
[0056]
  The condition of the vehicle state is described by the expression CSTATE (x), and whether or not the condition is met by substituting the output (48 bits) of the vehicle state detection means 901 for the value (x) is a true / false value. The logical expression to be output.
[0057]
  FIG. 11 is a flowchart showing the operation of the detection result processing means 1001. First, in step 1101, it is determined whether or not a detection result to be processed remains. In step 1102, the detection result is read, and the detected target name (named NAME) and coordinates are read. In step 1103, a presentation pattern whose target name is NAME is acquired from the presentation pattern storage table. In step 1104, it is checked whether or not the vehicle state obtained from the vehicle state detection means 901 satisfies the vehicle state conditional expression in the presentation pattern obtained in step 1103. If the conditional expression is satisfied, the process proceeds to step 1105. If not satisfied, the process proceeds to step 1101. In step 1105, data necessary for the presentation pattern is generated, and data generated in the format designated by the presentation means 205 designated by the presentation pattern is output. In step 1105, the operation differs depending on the presentation pattern. The above procedure is repeated until there is no detection result.
[0058]
  Hereinafter, the operation of step 1105 differs depending on the detection target, and a specific example will be described.
[0059]
  □ Road sign "Height restriction (3.3m)"
  Detection result: (LIMIT-H33, 100, 120, 200, 225)
  Presentation pattern: (LIMIT-H33, CSTATE (x), SP, PAT1, PARAM1 = “There is a height limit”, PARAM2 = 1 (times))
  However,
  CSTATE (x) = (GETHEIGHT (x)> 3.3)
  This means that when the vehicle height is 3.3 m or more, “there is a height restriction” is once spoken.
[0060]
  □ Road sign “Width limit (2.2m)”
  Detection result: (LIMIT-W22, 100, 120, 200, 225)
  Presentation pattern: (LIMIT-W22, CSTATE (x), SP, PAT1, PARAM1 = "There is a width limit", PARAM2 = 1 (times))
  However,
  CSTATE (x) = (GETWIDTH (x)> 2.2)
  When the vehicle width is 2.2 m or more, this means that “there is a width limitation” once.
[0061]
  Note that the in-vehicle device 107 of FIG. 9 includes a brake, an accelerator, a handle, a headlight, and the like, and these devices can be controlled according to the detection result. For example, if a “stop” sign is detected and the vehicle speed is 30 km / h or higher, the brake is turned on, or if the red signal is detected and the vehicle speed is 0 km / h (when stopped), the headlight is turned off. It is possible to do.
[0062]
  As described above, the detected content (road signs, traffic lights, etc.) is not always presented to the driver, but is presented in a presentation method suitable for the state of the car, causing confusion for the driver who concentrates on driving. It can support safe driving.
[0063]
  (Embodiment 3)
  Further, the safe driving support program and the presentation pattern storage table necessary for realizing the configuration of the safe driving support device described in each of the above embodiments are stored in another server computer via a communication network. By doing so, it is possible to download the processes shown in the above embodiments from the server computer and implement them.
[0064]
  FIG. 12 is a diagram for explaining a case where the safe driving support process of the first and second embodiments is performed in cooperation with the server computer 1203 that stores the safe driving support program and the presentation pattern storage table. The in-vehicle computer shown in the first or second embodiment is further provided with transmission / reception means 1201 for communicating with the server computer 1203 outside the vehicle via the communication network 1202.
[0065]
  The in-vehicle computer 1200 downloads the program and the presentation pattern storage table from the server computer 1203, stores them in the storage means 103, and executes the safe driving support process using the CPU 104 as in the first and second embodiments.
[0066]
  Alternatively, the input image input by the image input unit 102 and the vehicle state input from the vehicle state detection unit 901 are transmitted to the server computer 1203 using the transmission / reception unit 1201, and the safe driving support process is executed on the server computer 1203. The result (presentation data corresponding to the presentation pattern) can be received by the in-vehicle computer 1200 and presented using the speaker 105, the display unit 106, and the in-vehicle device 107.
[0067]
  By configuring in this way, the developer of the safe driving support program can always provide the latest program or can be processed using the calculation resources of the server computer 1203, so that even when the performance of the in-vehicle computer 1200 is low, the program is stable. Thus, it is possible to provide a function for assisting safe driving.
[0068]
  The storage means 103 of the above embodiment can be similarly implemented as long as it can record a program, such as a floppy disk, an optical disk, an IC card, a ROM cassette, a solid memory, a hard disk, or the like.
[0069]
  In the above embodiment, the image input means (on-vehicle camera) captures the front of the car. However, the present invention is not limited to this, and the image input means (on-vehicle camera) may be installed so as to capture the rear and lateral directions. As an advantage when photographing the rear, it is possible to detect a destination display board installed on the opposite lane side, so that it is possible to check the destination of each direction at the intersection that has just passed.
[0070]
  In the above embodiment, the presentation time is specified by a fixed time (seconds), but “present until a detection result is obtained from an image of the next N frame (N is a natural number)”. It is also possible. In this case, the parameter of the presentation pattern indicating the presentation time is the number of frames (N). This is to cope with the case where the detection time of the detection target after the image is input is not always constant.
[0071]
  Further, although the position in the detection result output by the image detection unit 202 is a circumscribed rectangle, the position is not limited to this and may be a polygon including the detection target.
[0072]
  In the above embodiment, the parameter values (presentation time and the like) of each output format of the presentation unit 205 are appropriately determined, but the present invention is not limited to this, and other values may be used.
[0073]
    As described above, according to this embodiment,,A safe driving support apparatus that includes an image detection unit that detects a detection target from an image and presents the presence of the detection target to a driver using a display unit, and uses region expansion information prepared in advance for each detection target Then, an area image obtained by enlarging the detection area to be detected based on the area enlargement information is obtained, and an image of the enlarged area portion is presented on the display means, thereby enabling safe driving.
[0074]
Preferably, the detection target is a traffic light, and the area enlargement information includes an amount of enlargement of the area below the traffic light, obtains an area image obtained by enlarging the detection area based on the area enlargement information, and the enlargement An image of the area portion that has been displayed on the display means,Since not only traffic lights but also auxiliary signals (signals such as “→”) can be displayed at the same time, it is possible to concentrate on safe driving without having to stare at actual small auxiliary signals.
[0075]
  Furthermore, it is preferable that an image of the enlarged area portion is presented to the display means only when the traffic light is a red signal, and only when the auxiliary signal (a signal such as “→”) is lit up. SignalSo you do n’t have to stare at the actual small auxiliary signal,In addition, it is not necessary to keep an eye on the display means every time it passes through a traffic light other than a red signal.Focus on safe driving.
[0076]
  From an image input means for inputting an image, an image detection means for detecting a road traffic sign or traffic light as a detection target from an image input from the image input means, and a storage means for storing presentation pattern information related to the road traffic sign or traffic light in advance Presentation means for selecting a presentation content from the acquired presentation pattern information and a detection target detected by the image detection means, wherein the presentation pattern information includes area expansion information, and the detection target is based on the area expansion information This is an image processing device that presents on the display means an image of the area portion where the detection area is enlarged, and can display not only the road traffic signs or traffic lights but also auxiliary signals (signals such as “→”).
[0077]
Further, by realizing the image detection process as a program for causing a computer to perform the process, the program can be updated to the latest one via a storage medium or a communication network, or can be executed by an arbitrary computer.
[0078]
【The invention's effect】
  According to the present invention, an object to be detected from an imageIs a traffic lightA safe driving support apparatus that includes an image detection means for detecting the presence of the detection target and presents the presence of the detection target to a driver using a display means.Enlarge the area below the traffic light that isAn area obtained by enlarging the detection area to be detected based on the area enlargement information using area enlargement informationimageOf the enlarged region portion ofPictureAn image is presented on the display means, and safe driving is possible.
[Brief description of the drawings]
FIG. 1 shows an example of a system according to the present invention.
FIG. 2 is a diagram illustrating a processing configuration inside a CPU according to the first embodiment;
FIG. 3 is a diagram for explaining a detection result of an image detection unit.
FIG. 4 is a diagram for explaining a presentation pattern
FIG. 5 is a flowchart showing the operation of the detection result processing means in the first embodiment.
FIG. 6 is an explanatory diagram of a presentation pattern in which a detection area is enlarged and displayed when a traffic light is detected.
FIG. 7 is an explanatory diagram of a presentation pattern that displays an enlarged detection area when a road traffic sign is detected.
FIG. 8 is a timing chart of image input, object detection and presentation.
FIG. 9 is a diagram showing another example of the system according to the present invention.
FIG. 10 is a diagram showing a processing configuration inside a CPU according to the second embodiment;
FIG. 11 is a flowchart showing the operation of the detection result processing means in the second embodiment.
FIG. 12 is an explanatory diagram of a data storage medium for storing a program for realizing the safe driving support apparatus of each of the embodiments by a computer system.
FIG. 13 is a processing configuration diagram of the prior art.
FIG. 14 is a diagram for explaining a prior art presentation method;
[Explanation of symbols]
  101 cars
  102 Image input means
  103 storage means
  104 CPU
  105 Speaker
  106 Display means
  107 In-vehicle equipment
  108 Input means
  201 Image buffer
  202 Image detection means
  203 Detection result processing means
  204 Presentation pattern storage table
  205 Presentation means
  901 Vehicle state detection means
  1001 Detection result processing means
  1002 Presentation pattern storage table
  1200 Onboard computer
  1201 Transmission / reception means
  1202 Communication network
  1203 server computer

Claims (5)

A safe driving support apparatus having an image detection means for detecting a traffic signal as a detection target from an image, and presenting the presence of the detection target to a driver using a display means, in a downward direction of the traffic light as the detection target and characterized by using a region growing information to enlarge the area, give the enlarged region image based on the detection area of the detection target in the area expansion information, and presents the images of the enlarged area portion on the display means Safe driving support device. Safe driving support apparatus according to claim 1, wherein the presenting an image of the enlarged area portion only if the traffic signal is red on the display means. An image input means for inputting an image;
Image detection means for detecting a road traffic sign or traffic light as a detection target from an image input from the image input means;
A presentation means for selecting the presentation content from the presentation pattern information acquired from the storage means preliminarily storing the presentation pattern information relating to road traffic signs or traffic lights and the detection target detected by the image detection means;
The presentation pattern information has a region growing information including the rate information to direction and expanded to expand, presented on the display means an image of the detection region enlarged area portions of the detection target based on the region growing information Image processing device. An image input means for inputting an image;
Image detection means for detecting a road traffic sign or traffic light as a detection target from an image input from the image input means;
A presentation means for selecting the presentation content from the presentation pattern information acquired from the storage means preliminarily storing the presentation pattern information relating to road traffic signs or traffic lights and the detection target detected by the image detection means;
The presentation pattern information, the presentation time has information and area expanding information on an image processing apparatus for presenting on the display means an image of the detected enlarged area portion detection area of the object based on the region growing information. The image processing apparatus according to claim 3, wherein the presentation pattern information includes information related to a presentation time.

Images