JP4183012B2 - Image processing apparatus and recording medium - Google Patents

Description

  The present invention relates to an image processing apparatus and a recording medium that execute various processes based on captured image data.

  As an image processing method for the captured image data in the image processing apparatus, there are the following methods.

  Image data is taken into the image processing device by an image input device such as a CCD camera incorporated in or externally connected to the image processing device such as a portable terminal, and the image data taken on the device is processed and edited.

For example, the captured image data is converted into a file or converted into a format that can be handled on the image processing apparatus, and an image database for use with specific software is created. In addition, it is converted into electronic data such as text information in combination with OCR software or the like.
In addition, there is a technique for extracting feature data by capturing an image and searching for feature data most similar to the captured feature data among registered feature data.
Japanese Patent Laid-Open No. 06-217037

  However, in such an image processing method, in order to utilize the captured image data, first, primary processing such as conversion of the image data is necessary, and it is difficult to maintain continuity with subsequent processing. Met.

  An object of the present invention is to provide an image processing apparatus and a recording medium capable of continuously executing image data processing, system control, and the like on the image processing apparatus side without requiring primary processing such as conversion by the user. Is to provide.

An image processing apparatus according to a first aspect of the present invention is an image processing apparatus that executes a predetermined process based on an image captured by an imaging unit, and includes an arbitrary application that is activated in correspondence with the image. Storage means for storing in advance and processing executed by this application in advance corresponding to a time zone, an image photographed by the photographing means when the application is started, and an image preliminarily stored in the storage means When the determination means for obtaining the approximate image and the determination means determine the approximate image, the application stored in the storage means in association with the image is started. Furthermore, when the application is started up, the time when the image capturing is performed corresponds to any of the stored time zones. Or to determine the, and control means for executing processing stored in correspondence to the time zone applicable.

The invention described in claim 2 is a recording medium storing a program for causing a computer to function as each means of the invention described in claim 1. For example, it corresponds to the storage medium 8 described in the embodiment of the invention.

According to the present invention, it is possible to start a predetermined application corresponding to a photographed image and execute processing according to the photographed time by a simple operation of photographing.

  Hereinafter, embodiments will be described in detail with reference to the drawings.

(First embodiment)
FIG. 1 is a diagram showing a mobile terminal 10 to which an image processing apparatus of the present invention is applied and a camera 20 connected externally. An image captured by the camera 20 is taken into the mobile terminal 10 and various image processing is performed by the mobile terminal 10.

  The internal configuration of the mobile terminal 10 will be described based on the block diagram shown in FIG. In the figure, a mobile terminal 10 is composed of a CPU 1, an input unit 2, a RAM 3, a transmission control unit 4, a display unit 5, a camera control unit 6, a storage device 7, and a storage medium 8. It is connected to the bus 9.

  The CPU 1 reads out various processing programs stored in the storage medium 8, temporarily stores them in the RAM 3, executes various processes based on the various processing programs, stores the processing results in the RAM 3, and displays them on the display unit 5. Let

  Further, the CPU 1 executes an image memory setting process (see FIG. 7) described later. In this image memory setting process, the CPU 1 stores the image data captured by the camera 20 in association with information such as the name of the application that is activated when the image is captured and the database name used at that time. The image is stored in the medium 8 and an image database is created.

  Further, the CPU 1 executes a corresponding AP execution process (see FIG. 8) described later. In the corresponding AP execution process, the CPU 1 activates a corresponding application based on the image database in the storage medium 8 from the image data captured by the camera 20, and executes various processes.

  The input unit 2 includes a keyboard having cursor keys, numeric input keys, various function keys, and the like, and a mouse that is a pointing device. The input unit 2 outputs a key press signal pressed by the keyboard to the CPU 1 and is operated by the mouse. A signal is output to CPU1. The user performs selection / instruction in various processes via the input unit 2.

  The RAM 3 has a program storage area for storing various processing programs and various application programs executed by the CPU 1, and a work area for temporarily storing input instructions, input data, and processing results.

  The transmission control unit 4 is composed of a modem (MODEM / DEModulator), a terminal adapter (TA), or the like, and performs communication with an external device via a communication line such as a telephone line or an ISDN line. Take control. In order to communicate with an external device such as a personal computer via a telephone line, the modem modulates digital data processed by the CPU 1 into an analog signal suitable for the frequency band of the telephone line, and also via the telephone line. The terminal adapter converts the existing interface into an ISDN-compatible interface for communication with external devices such as personal computers via the ISDN line. It is a device to do.

  The display unit 5 is configured by a liquid crystal display device or the like, and displays a processing result and the like based on output data input from the CPU 1.

  In response to an instruction from the CPU 1, the camera control unit 6 converts a captured image captured by the camera 20 into image data corresponding to the mobile terminal 10 and outputs the image data to the CPU 1.

  The storage device 7 has a storage medium 8 in which programs, data, and the like are stored in advance, and the storage medium 8 is constituted by a magnetic or optical recording medium or a semiconductor memory. The storage medium 8 is fixedly attached to the storage device 7 or is detachably mounted. The storage medium 8 includes various application programs corresponding to the mobile terminal 10 and an image memory setting processing program (described later). 7), a corresponding AP execution processing program (see FIG. 8), and data, documents, etc. processed by each processing program are stored.

  The program, data, etc. stored in the storage medium 8 are partly or entirely received from the transmission control unit 4 via a transmission medium such as a communication line from another device such as a server or a client and stored. In addition, the storage medium 8 may be a storage medium of a server constructed on a network. Furthermore, the program may be transmitted to a server or client via a transmission medium such as a communication line and installed in these devices.

  Further, as shown in FIG. 3, the storage medium 8 has a memory area for storing various memories and files.

  In the figure, an image memory 81 stores a plurality of image data as shown in FIG. Each image data includes information such as an image number, an application (AP) name that is activated when the image data is selected, a database (DB) name used in the application, and a record number. Stored in association. If there is no database used by the activated application for this image data, the database name and record number are not set.

  In FIG. 3, the database file 82 stores a plurality of databases such as a product database and a specification database together with application names to be used, as shown in FIG.

  In each database, for example, in a product database, a product name and a plurality of data records of the product information are associated with a record number, and in a specification database, a specification name and a plurality of data records of the specification information are associated with a record number. Stored.

  In FIG. 3, the application file 83 stores a plurality of application names in association with software names to be used, as shown in FIG.

  Details of the subsequent mail setting file 84, Internet setting file 85, etc. will be described later.

  Next, the operation will be described. The operator needs to take an image of the target product in advance, create an image database together with each product database, and store it in the mobile terminal 10. The operation of the portable terminal 10 in the image memory setting process for creating and saving the image database will be described based on the flowchart shown in FIG.

  Here, a program for realizing each function described in these flowcharts is stored in the storage medium 9 in the form of a readable program code, and the CPU 1 sequentially performs operations according to the program code. Execute. In addition, the CPU 1 can sequentially execute the operation according to the above-described program code transmitted via the communication line. In other words, in addition to the storage medium 9, it is also possible to execute operations specific to this embodiment by using programs and data supplied externally via a communication line.

  First, a target product image is photographed by the camera 20 externally connected to the mobile terminal 10 (step S1). The captured image captured by the camera 20 is converted into image data by the camera control unit 6 and output to the CPU 1. Then, the CPU 1 temporarily stores the input image data in the RAM 3 and displays it on the display unit 5 (step S2).

  When an application to be activated corresponding to the image data is selected / instructed by an instruction from the input unit 2 (step S3), the CPU 1 temporarily stores the selected application name in the RAM 3 and refers to the image memory 81. Then, it is determined whether or not a database to be used exists in the application (step S4).

  When there is a database (step S4: YES), the CPU 1 refers to the database file 82 in the storage medium 8, selects the database used by the selected application (step S5), and temporarily stores the database name in the RAM 3. The data is stored and displayed on the display unit 5 (step S6).

  Then, it is searched whether or not a data record used by the selected application is stored in the database. When the data record is not stored (step 7: YES), the CPU 1 causes the input unit 2 to input the target data, adds the data record to the database, stores the data record in the storage medium 8, and the record. No. is temporarily stored in the RAM 3 (step S8).

  If the target data record is already stored in the database (step S7: NO), the CPU 1 reads the record number and temporarily stores it in the RAM 3 (step S9).

  When it is confirmed by the instruction from the input unit 2 that the captured image data is to be added to the image database (step S10), the CPU 1 stores the image data in the application name, database name, In addition, it is stored in the image memory 81 in association with information such as a record number (step S11). When the above processing is performed, the CPU 1 ends the image memory setting processing.

  If the captured image data is not added to the image database (step S10: NO), the CPU 1 initializes the settings of the image data, application name, database name, etc. temporarily stored in the RAM 3 (step S13). The image memory setting process ends.

  In step S4, when there is no database used by the selected application (step S4: NO), the CPU 1 confirms that the image data is confirmed and added to the image database according to an instruction from the input unit 2. (Step S14: YES), the selected application name is associated with the image data and additionally stored in the image memory 81 (step S15), and the image memory setting process is terminated. In this case, the corresponding database name and record number are not set.

  If the captured image data is not added to the image database (step S14: NO), the CPU 1 initializes the settings of image data, application name, etc. temporarily stored in the RAM 3 (step S17), and sets the actual image memory. The process ends.

  As described above, captured image data is associated with each piece of information and stored in the image memory 81 as an image database. The image memory setting process is appropriately executed as necessary, and the contents of the image memory 81 are added / updated.

  When the user photographs the target product with the camera 20, the CPU 1 in the mobile terminal 10 reads the corresponding AP execution processing program from the storage medium 8 and executes the corresponding AP execution processing. The operation of the mobile terminal 10 in the corresponding AP execution process will be described based on the flowchart shown in FIG.

  The captured image captured by the camera 20 is converted into image data by the camera control unit 6 and output to the CPU 1. The CPU 1 displays the input image data on the display unit 5 and temporarily stores it in the RAM 3 (step S21).

  Next, the CPU 1 reads the first image data from the image memory 81 in the storage medium 8 (step S23). Then, the read image data and the captured image data are compared to determine whether they are approximate (step S24).

  If not approximate (step S25: NO), the CPU 1 reads the next image data from the image memory 81 (step S28), and repeats the determination of approximation with the photographed image data (step S24).

  If they are approximate (step S25: YES), the CPU 1 adds the image number of the read image data to the approximate image data list display (step S26).

  When the comparison of all image data in the image memory 81 with the captured image data is completed (step S27: YES), the CPU 1 causes the display unit 5 to display an approximate image data list display.

  When appropriate image data is selected / instructed from the displayed image data list display by the input unit 2 (step S29), the CPU 1 refers to the image memory 81 and the application file 83 in the storage medium 8, An application corresponding to the selected image data is activated (step S30).

  If there is a database corresponding to the activated application (step S31: YES), the CPU 1 refers to the database file 82 in the storage medium 8 and reads the record data of the corresponding record No. in the corresponding database. Is displayed on the display unit 5 (step S32). When this application process ends, the CPU 1 ends the corresponding AP execution process.

  As described above, when a product is photographed by the user with the camera 20, the photographed image is collated with the image data in the image database in the mobile terminal 10, and the corresponding application is automatically started. Therefore, the user can easily know the detailed information of the product simply by photographing the product, and the product information can be searched by a simple operation.

(Second Embodiment)
In addition to the first embodiment, the processing to be executed can be changed according to the time zone when the target product is photographed. In the second embodiment, a mail setting process for selecting mail or FAX according to a time zone when mail software is started as an application will be described.

  First, the configuration will be described. Since the portable terminal in the second embodiment has the same configuration as the portable terminal 10 (see FIG. 2) described in the first embodiment, the same reference numerals are given and the description thereof is omitted. .

  The storage medium 8 in the portable terminal 10 further stores a mail setting file 84 shown in FIG. 9 and a mail setting processing program (see FIG. 10) described later.

  In FIG. 9, the mail setting file 84 stores a mail address and a FAX number in association with each time zone for each image number of image data stored in the image memory 81.

  Next, the operation will be described. When mail software is activated as a corresponding application process in step S30 of FIG. 8 (see FIG. 6), the CPU 1 in the mobile terminal 10 reads the mail setting process program from the storage medium 8 and executes the mail setting process. . The operation of the mobile terminal 10 in this mail setting process will be described based on the flowchart shown in FIG.

  First, when the mail software is activated, the CPU 1 displays an initial main screen on the display unit 5 (step S41).

  When the mail data created from the input unit 2 is input, the CPU 1 searches the mail setting file 84 in the storage medium 8 and data corresponding to the image number of the image data temporarily stored in the RAM 3. It is searched whether or not a record is set (step S43).

  When the data record corresponding to the image data is set (step S43: YES), the CPU 1 acquires the current time from the clock in the mobile terminal 10 (step S44). Then, from the data record corresponding to the image data, the transmission destination type (that is, mail or FAX) set in the time zone including the acquired current time is determined (step S46).

  If the determined transmission destination type is mail (step S46: mail), the CPU 1 sets the mail address as the transmission destination and causes the display unit 5 to display that fact (step S47). When mail transmission is instructed via the input unit 2 (step S48: YES), the CPU 1 transmits mail data to the set transmission destination (step S49), and the mail setting process is terminated.

  If an instruction to cancel mail transmission is given via the input unit 2 (step S50: YES), the CPU 1 initializes the setting of the transmission destination and ends the mail setting process.

  If the destination type determined in step S46 is FAX (step S46: FAX), the CPU 1 converts the created mail data into FAX data corresponding to FAX (step S53). Then, the set FAX No. is set as the transmission destination and the fact is displayed on the display unit 5 (step S54).

  Then, when FAX transmission is instructed via the input unit 2 (step S55: YES), the CPU 1 transmits FAX data to the set transmission destination (step S56), and ends the mail setting process.

  In addition, when cancellation of FAX transmission is instructed via the input unit 2 (step S57: YES), the CPU 1 initializes transmission destination settings (step S58) and ends the mail setting process.

  In step S43, as a result of searching the mail setting file 84 in the storage medium 8, no data record corresponding to the image number of the image data temporarily stored in the RAM 3 is set (step S43: NO). Alternatively, in step S45, when the time zone including the acquired current time is not set in the data record corresponding to the image data (step S45: NO), the CPU 1 displays that fact on the display unit 5. Then, the destination of the created mail data is input / set from the input unit 2 (step S52).

  Thereafter, the CPU 1 transmits the mail data created to the set transmission destination (step S49), and ends the mail setting process.

  As described above, when the mail software is activated, whether the created mail data is transmitted by mail or by FAX is automatically selected according to the time zone. Therefore, mail data created by mail or FAX can be transmitted at the convenience of the destination.

  Although either mail or FAX is set as the transmission destination, it can be configured to transmit to both mail and FAX by changing the data record of the mail setting file 84.

(Third embodiment)
Further, in addition to the first embodiment or the second embodiment, the processing to be further executed can be changed according to the time zone when the target product is photographed. In the third embodiment, an internet setting process for selecting a home page to be displayed according to a time zone when browser software is started as an application will be described.

  First, the configuration will be described. Since the portable terminal in the third embodiment has the same configuration as the portable terminal 10 (see FIG. 2) described in the first embodiment and the second embodiment, the same reference numerals are used. A description thereof will be omitted.

  The storage medium 9 in the mobile terminal 10 further stores an Internet setting file 85 shown in FIG. 11 and an Internet setting processing program (see FIG. 12) described later.

  In FIG. 11, the Internet setting file 85 stores a URL in association with a time zone for each image number of image data stored in the image memory 81.

  Next, the operation will be described. In step S30 of FIG. 8, when the browser software is started as the corresponding application process (see FIG. 6), the CPU 1 in the mobile terminal 10 reads the Internet setting process program from the storage medium 8 and executes the Internet setting process. . The operation of the mobile terminal 10 in this Internet setting process will be described based on the flowchart shown in FIG.

  When the browser software is started as an application, the CPU 1 searches the Internet setting file 85 in the storage medium 8 and determines whether a data record corresponding to the image number of the image data temporarily stored in the RAM 3 is set. Search is performed (step S61).

  When the data record corresponding to the image data is set (step S61: YES), the CPU 1 acquires the current time from the clock in the mobile terminal 10 (step S62). And it is discriminate | determined from the data record corresponding to the image data whether URL is set in the time slot | zone containing the acquired present time (step S63).

  If the URL has been set (step S63: YES), the CPU 1 sets the URL as a connection destination, displays that fact on the display unit 5 (step S64), and connects to the provider (step S68).

  If no data record corresponding to the image data is set in step S61 (step S61: NO), or in step S63, the URL of the time zone including the current time acquired in the corresponding data record is set. If not (step S63: NO), the CPU 1 causes the display unit 5 to display an initial menu screen (step S65), and causes the input unit 2 to input and set a URL (step S66).

  When the set URL is confirmed through the input unit 2 (step S67: YES), the CPU 1 connects to the provider (step S68). When the set URL is not confirmed (step 67: NO), the CPU 1 causes the input unit 2 to input and set the URL again (step S66).

  When connected to the provider, the CPU 1 transmits the set URL to the provider (step S69), displays the received data received on the display unit 5 (steps S70 and S71), and ends the Internet setting process. .

  As described above, when the browser software is activated, the homepage address to be displayed is automatically selected according to the time zone. Therefore, by setting in advance a homepage address that matches the time zone, the homepage desired by the user can be easily displayed.

  In the first to third embodiments, the camera 20 is externally connected. However, the camera 20 may be built in the mobile terminal 10 as a matter of course.

(Fourth embodiment)
In the portable terminal to which the image processing apparatus of the present invention is applied, when the photographed image by the camera is displayed on the display unit of the portable terminal, the display form of the photographed image can be changed by moving the camera or the like. In the fourth embodiment, a display screen control process for controlling a photographed image displayed on the display unit according to the photographing form of the camera will be described.

  First, the configuration will be described. Since the mobile terminal 10 in the fourth embodiment has the same configuration as that of the mobile terminal 10 (see FIG. 2) described in the first to third embodiments, the same reference numerals are given. Detailed description will be omitted.

  The mobile terminal 10 and the externally connected camera 20 are each provided with a detection unit for detecting the position, moving direction, moving amount, and the like of the camera 20 with respect to the mobile terminal 10. The detection unit 10 a provided in the mobile terminal 10 detects information such as the position, movement direction, movement amount, and volume of the camera 20 with respect to the mobile terminal 10, and outputs the detection information to the CPU 1.

  Further, the storage medium 8 of the portable terminal 10 further includes a display position file (FIG. 13A), a movement direction file (FIG. 13B), a movement amount file (FIG. 13C), A size file ((d) in the figure) and a display screen control processing program (see FIG. 14) described later are stored.

  13, the display position file 86 (FIG. 13A) stores the position of the camera 20 relative to the mobile terminal 10 and the position of the captured image displayed on the display unit 5 in association with each other, and the movement direction file 87 ( (B)) stores the position of the camera 20 with respect to the mobile terminal 10, the captured image displayed on the display unit 5, and the movement direction in association with each other.

  Further, the movement amount file 88 ((c) in the figure) stores the acceleration of the camera 20 with respect to the portable terminal 10 and the movement amount of the captured image displayed on the display unit 5 in association with each other, and the size file 89 (in the same figure). (D)) stores the volume difference between the mobile terminal 10 and the camera 20 and the magnification of the captured image displayed on the display unit 5 in association with each other.

  Next, the operation will be described. The CPU 1 in the portable terminal 10 reads the display screen control processing program from the storage medium 8 at regular intervals and executes the display screen control processing. The operation of the mobile terminal 10 in this display screen control process will be described based on the flowchart shown in FIG.

  First, the detection part 10a provided in the portable terminal 10 detects the position information of the camera 20 with respect to the portable terminal 10, and outputs it to CPU1 (step S81). Then, the CPU 1 refers to the display position file in the storage medium 8 from the input position information, and if the position of the camera 20 is on the left side with respect to the mobile terminal 10 (step S82: left), the captured image is displayed on the left side. And display on the display unit 5 (step S83).

  If the detected position of the camera 20 is on the right side with respect to the portable terminal 10 (step S82: right), the CPU 1 sets the captured image to be displayed on the right side and displays it on the display unit 5 (step S82). S84).

  Next, the detection unit 10a detects acceleration information of the camera 20 with respect to the mobile terminal 10, and outputs the acceleration information to the CPU 1 (step S85). Then, when the CPU 1 calculates the movement direction and movement amount of the camera 20 relative to the mobile terminal 10 from the detected acceleration information (step S86), the CPU 1 refers to the movement direction file in the storage medium 8 and starts from the calculated movement direction. The direction in which the captured image is moved is set (step S87).

  Further, the CPU 1 refers to the movement amount file in the storage medium 8 and sets the amount by which the captured image is moved from the calculated movement amount (step S88). Then, the captured image is moved based on the set moving direction and moving amount and displayed on the display screen (step S89).

  Subsequently, when the CPU 1 detects the volume information of the mobile terminal 10 from the detection unit 10a provided in the mobile terminal 10 and the volume information of the camera 20 from the detection unit provided in the camera 20 (step S90). The amount of difference between the detected volume information is calculated.

  Then, referring to the size file in the storage medium 8, the screen size to be changed is set (step S91). Thereafter, the CPU 1 changes the size of the shooting screen based on the set screen size and displays it on the display screen (step S92).

  When the setting of the captured image is changed / displayed as described above, the CPU 1 ends the display screen control process.

  As described above, the position and size of the captured image displayed on the display unit 5 of the mobile terminal 10 are controlled by the detection information detected by the detection units provided in the mobile terminal 10 and the camera 20. Thus, the user can easily change the display form of the photographed image displayed on the display unit 5 by operating the photographing form of the camera 20.

  In addition, in this invention, it can also comprise so that a process may be changed with the imaging | photography direction of the imaging | photography object goods.

  In the present invention, the object to be photographed is a product, but a person may be photographed. In that case, since the set person image may differ from the person's clothes taken every time, the skin color (face portion) may be detected to identify the person. By photographing and specifying a plurality of persons, mail, FAX, etc. can be collectively transmitted to a plurality of destinations.

It is a figure which shows the portable terminal 10 and the camera 20 to which this invention is applied. 2 is a block diagram showing an internal configuration of a mobile terminal 10. FIG. 3 is a diagram showing a memory area in the storage medium 8. FIG. 3 is a diagram showing an image memory 81. FIG. It is a figure which shows the database file. It is a figure which shows the application file 83. FIG. 6 is a flowchart illustrating image memory setting processing in the first embodiment. It is a flowchart which shows corresponding AP execution processing in 1st Embodiment. It is a figure which shows the mail setting file 84. FIG. It is a flowchart which shows the mail setting process in 2nd Embodiment. It is a figure which shows the internet setting file 85. FIG. It is a flowchart which shows the internet setting process in 3rd Embodiment. The figure which shows the display position memory 86 (the figure (a)), the movement direction memory 87 (the figure (b)), the movement amount memory 88 (the figure (c)), and the size memory 89 (the figure (d)). It is. It is a flowchart which shows the display screen control process in 4th Embodiment.

Explanation of symbols

10 Mobile terminal 1 CPU
2 Input unit 3 RAM
4 Transmission Control Unit 5 Display Unit 6 Camera Control Unit 7 Storage Device 8 Storage Medium 81 Image Memory 82 Database File 83 Application File 84 Mail Setting File 85 Internet Setting File 86 Display Position Memory 87 Movement Direction Memory 88 Movement Amount Memory 89 Size Memory 9 Bus 10a detector 20 Camera 20a detector

Claims (2)

An image processing apparatus that executes predetermined processing based on an image photographed by a photographing means,
A storage unit that stores in advance an arbitrary application that is activated in correspondence with the image and that stores a process executed by the application in advance corresponding to a time zone ;
A determination unit that compares an image captured by the imaging unit with an image stored in advance in the storage unit to obtain an approximate image when the application is started ;
When the approximate image is determined by the determination unit, the application stored in the storage unit is activated in association with the image, and further, the image is captured by the imaging unit when the application is activated. An image processing apparatus comprising: control means for determining which of the stored time zones corresponds to the time and executing a process stored corresponding to the corresponding time zone . Computer
Means for preliminarily storing in the storage means an arbitrary application activated in correspondence with the image, and storing in advance in the storage means processing executed by this application in correspondence with a time zone;
A determination unit that compares an image captured by the imaging unit with an image stored in advance in the storage unit to obtain an approximate image when the application is started;
When the approximate image is determined by the determination unit, the application stored in the storage unit is activated in association with the image, and further, the image is captured by the imaging unit when the application is activated. Control means for determining which of the stored time zones corresponds to the time and executing the stored process corresponding to the corresponding time zone
A computer-readable recording medium storing a program for functioning as a computer .

Images