CN117807344A - Method and device for constructing IP database - Google Patents
Method and device for constructing IP database Download PDFInfo
- Publication number
- CN117807344A CN117807344A CN202311863945.1A CN202311863945A CN117807344A CN 117807344 A CN117807344 A CN 117807344A CN 202311863945 A CN202311863945 A CN 202311863945A CN 117807344 A CN117807344 A CN 117807344A
- Authority
- CN
- China
- Prior art keywords
- address
- database
- data
- constructing
- addresses
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/90—Details of database functions independent of the retrieved data types
- G06F16/95—Retrieval from the web
- G06F16/958—Organisation or management of web site content, e.g. publishing, maintaining pages or automatic linking
- G06F16/986—Document structures and storage, e.g. HTML extensions
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/81—Indexing, e.g. XML tags; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/80—Information retrieval; Database structures therefor; File system structures therefor of semi-structured data, e.g. markup language structured data such as SGML, XML or HTML
- G06F16/83—Querying
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The embodiment of the application provides a method and a device for constructing an IP database, wherein the method comprises the following steps: the analysis results corresponding to all the IP addresses are obtained from the IP library server; aiming at the analysis result corresponding to each IP address, converting according to a preset data structure to obtain the record data of the IP address; and converting the recorded data of the IP address into a key value pair form and storing the key value pair form in an IP database. The method comprises the steps of periodically traversing the analysis results corresponding to the IP addresses in the whole IP library server to obtain analysis results corresponding to all the IP addresses, and then storing the data into a high-performance high-availability IP database by using a technical means to provide IP address analysis service for the service in real time by using the IP database, thereby achieving the purpose of avoiding the need of analyzing the IP addresses by requesting a server interface in real time, reducing the delay of IP address analysis and improving the bearing capacity of the system.
Description
Technical Field
The embodiment of the application relates to the technical field of internet, in particular to a method, a device, computer equipment and a computer readable storage medium for constructing an IP database.
Background
IP address resolution is a very basic function in internet services. The current industry analyzes the IP address by calling the HTTP interface of the IP library server in real time to perform data service, namely, returning the analysis result of requesting the IP address in real time through the interface according to the rule of Token using time charging.
Although specific analysis results of the IP address, such as province, city, isp and other information, can be obtained by calling the HTTP interface of the IP library server in real time. However, in the practical process, the manner of calling the IP library in real time to obtain the analysis result of the IP address has the following disadvantages: (1) the mode of accessing the interface in real time cannot hold high concurrent flow requests, (2) access through the real-time interface can cause the increase of the delay of the whole service link.
Disclosure of Invention
An object of an embodiment of the present application is to provide a method, an apparatus, a computer device, and a computer readable storage medium for constructing an IP database, for solving the following problems: the manner of acquiring the analysis result of the IP address by calling the IP library in real time cannot process the high concurrent flow request, and the delay of the overall service link is increased.
An aspect of an embodiment of the present application provides a method for constructing an IP database, the method including:
the analysis results corresponding to all the IP addresses are obtained from the IP library server;
aiming at the analysis result corresponding to each IP address, converting according to a preset data structure to obtain the record data of the IP address;
and converting the recorded data of the IP address into a key value pair form and storing the key value pair form in an IP database.
Optionally, the obtaining the analysis results corresponding to all the IP addresses from the IP library server includes:
and obtaining analysis results corresponding to all the IP addresses from the IP library server according to a preset period.
Optionally, after the step of converting the analysis result corresponding to each IP address according to a preset data structure to obtain the record data of the IP address, the method further includes:
uploading the recorded data of the IP address to a file management system;
and reading the record data of the IP address from the file management system through an IP data storage system, and storing the record data of the IP address.
Optionally, the IP database is deployed in a Redis database system, and the converting the record data of the IP address into a key value pair form and storing the key value pair form in the IP database includes:
and reading the record data of the IP address from the IP data storage system through the Redis database system, converting the record data of the IP address into a key value pair form and storing the key value pair form in the IP database.
Optionally, the method further comprises:
receiving an IP address resolution request; wherein, the IP address analysis request carries the IP address to be analyzed;
and inquiring a target analysis result corresponding to the IP address to be analyzed from an IP database.
Optionally, the querying, from an IP database, a target resolution result corresponding to the IP address to be resolved includes:
and inquiring a target analysis result corresponding to the IP address to be analyzed from IP address record data stored in the IP database in the form of key value pairs.
Optionally, the method further comprises:
and accessing the data resources in the server corresponding to the IP address to be resolved according to the target resolution result.
Optionally, the method further comprises:
forwarding the IP address resolution request to an IP library server under the condition that the access to the server corresponding to the IP address to be resolved fails according to the target resolution result;
receiving a real-time analysis result corresponding to the IP address to be analyzed, which is returned by the IP library server;
and accessing a server corresponding to the IP address to be resolved according to the real-time resolving result.
An aspect of an embodiment of the present application further provides an apparatus for constructing an IP database, where the apparatus includes:
the data acquisition module is used for acquiring analysis results corresponding to all the IP addresses from the IP library server;
the data conversion module is used for converting the analysis result corresponding to each IP address according to a preset data structure to obtain the record data of the IP address;
and the data storage module is used for converting the record data of the IP address into a key value pair form and storing the key value pair form in an IP database.
An aspect of the embodiments of the present application further provides a computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the method of building an IP database as described above when the computer program is executed.
An aspect of the embodiments of the present application further provides a computer readable storage medium having stored therein a computer program executable by at least one processor to cause the at least one processor to perform the steps of the method of building an IP database as described above when the computer program is executed.
According to the method, the device, the equipment and the computer readable storage medium for constructing the IP database, the IP database service corresponding to the IP library server is deployed, the analysis results corresponding to the IP addresses in the whole IP library server are traversed periodically to obtain the analysis results corresponding to all the IP addresses, then the data are stored into the high-performance high-availability IP database by using the technical means, so that the IP address analysis service is provided for the service in real time by using the IP database, the aim of avoiding the need of analyzing the IP addresses by requesting the server interface in real time is fulfilled, the delay of IP address analysis is reduced, and meanwhile, the bearing capacity of the system is improved.
Drawings
FIG. 1 schematically illustrates an application environment diagram of a method of building an IP database according to an embodiment of the present application;
fig. 2 schematically shows a flowchart of a method of constructing an IP database according to an embodiment one of the present application;
FIG. 3 schematically illustrates a modular block diagram of a method of building an IP database according to an embodiment one of the present application;
fig. 4 schematically shows a block diagram of an apparatus for constructing an IP database according to a second embodiment of the present application; and
Fig. 5 schematically shows a hardware architecture diagram of a computer device adapted to implement a method of building an IP database according to a third embodiment of the present application.
Detailed Description
In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
It should be noted that the descriptions of "first," "second," etc. in the embodiments of the present application are for descriptive purposes only and are not to be construed as indicating or implying a relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defining "a first" or "a second" may explicitly or implicitly include at least one such feature. In addition, the technical solutions of the embodiments may be combined with each other, but it is necessary to base that the technical solutions can be realized by those skilled in the art, and when the technical solutions are contradictory or cannot be realized, the combination of the technical solutions should be regarded as not exist and not within the protection scope of the present application.
In the prior art, IP address resolution is a very basic function in internet services. The current industry analyzes the IP address by calling the HTTP interface of the IP library server in real time to perform data service, namely, returning the analysis result of requesting the IP address in real time through the interface according to the rule of Token using time charging.
Although specific analysis results of the IP address, such as province, city, isp and other information, can be obtained by calling the HTTP interface of the IP library server in real time. However, in the practical process, the manner of calling the IP library in real time to obtain the analysis result of the IP address has the following disadvantages: (1) the mode of accessing the interface in real time cannot hold high concurrent flow requests, (2) access through the real-time interface can cause the increase of the delay of the whole service link.
In view of this, the present application aims to provide a method for constructing an IP database, by acquiring the parsing results corresponding to all IP addresses from an IP library server; aiming at the analysis result corresponding to each IP address, converting according to a preset data structure to obtain the record data of the IP address; and converting the recorded data of the IP address into a key value pair form and storing the key value pair form in an IP database. Through deploying the off-line IP database service corresponding to the IP library server, the analysis results corresponding to the IP addresses in the whole IP library server are traversed periodically to obtain the analysis results corresponding to all the IP addresses, and then the data are stored into the high-performance and high-availability IP database by utilizing the technical means, so that the IP database is utilized to provide the IP address analysis service for the service in real time, the aim of avoiding the need of requesting the server interface in real time to analyze the IP addresses is achieved, the delay of IP address analysis is reduced, and meanwhile, the bearing capacity of the system is improved.
Various embodiments are provided to further introduce a solution for building an IP database, with particular reference to the following.
In the description of the present application, it should be understood that the numerical references before the steps do not identify the order of performing the steps, but are only used for convenience in describing the present application and distinguishing each step, and thus should not be construed as limiting the present application.
The following is a term explanation of the present application:
redis: is an open source, uses ANSI C language to write, support network, can be based on memory and can also be durable log type, key-Value database.
File system: is a system for naming files and placing logical storage and recovery of files.
IP address: the IP protocol provides a unified address format that assigns a logical address to each network and each host on the internet to mask differences in physical addresses.
IP library: and a data set formed by the analysis results of all the IP addresses.
Isps: internet Service Provider, an internet service provider, i.e. a telecommunications carrier that provides internet access services, information services, and value added services to a broad spectrum of users in a comprehensive manner.
Fig. 1 schematically shows an environmental application schematic according to an embodiment of the present application. As shown in fig. 1:
the computer device 10000 can be connected to the client 30000 via a network 20000.
The computer device 10000 can provide services such as network debugging, or return result data of building an IP database to the client 30000, or the like.
The computer device 10000 can be located in a data center such as a single venue or distributed in different geographic locations (e.g., in multiple venues). The computer device 10000 can provide services via one or more networks 20000. Network 20000 includes various network devices such as routers, switches, multiplexers, hubs, modems, bridges, repeaters, firewalls, proxy devices, and/or the like. Network 20000 may include physical links such as coaxial cable links, twisted pair cable links, fiber optic links, combinations thereof, and the like. Network 20000 may include wireless links such as cellular links, satellite links, wi-Fi links, and the like.
The computer device 10000 can be implemented by one or more computing nodes. One or more computing nodes may include virtualized computing instances. Virtualized computing instances may comprise emulation of virtual machines, e.g., computer systems, operating systems, servers, etc. The computing node may load the virtual machine by the computing node based on the virtual image and/or other data defining the particular software (e.g., operating system, dedicated application, server) used for the emulation. As the demand for different types of processing services changes, different virtual machines may be loaded and/or terminated on one or more computing nodes. A hypervisor may be implemented to manage the use of different virtual machines on the same computing node.
The client 30000 may be configured to access the content and services of the computer device 10000. Client 30000 can include any type of electronic device, such as a mobile device, tablet device, laptop computer, workstation, virtual reality device, gaming device, set top box, digital streaming media device, vehicle terminal, smart television, set top box, and the like.
The client 30000 may output (e.g., display, render, present) the result data of building the IP database, etc., to the user.
The network debugging scheme will be described below by way of various embodiments. The scheme may be implemented by the computer device 10000.
Example 1
Fig. 2 schematically shows a flowchart of a method of constructing an IP database according to an embodiment of the present application. Comprising steps S202-S206, wherein,
step S202, obtaining analysis results corresponding to all IP addresses from an IP library server;
the embodiment can be applied to the IP proxy service, and an IP database is constructed through the IP proxy service, so that when an IP address analysis request is received, an analysis result corresponding to the IP address is directly obtained from the constructed IP database, and the need of requesting data from an IP library server in real time is avoided.
Specifically, the web crawler technology can be utilized to periodically obtain the analysis results corresponding to all the IP addresses from the IP library server in an offline manner, so that the maintained IP database is updated by utilizing the analysis results corresponding to the obtained IP addresses.
Step S204, converting the analysis result corresponding to each IP address according to a preset data structure to obtain the record data of the IP address;
specifically, after the analysis results corresponding to all the IP addresses are obtained, the analysis result corresponding to each IP address may be converted according to a preset data structure to obtain record data of one IP address. The preset data structure may be a pre-designated data structure, so that the data can be conveniently queried, for example, assuming that the analysis result corresponding to the IP address includes information such as province, city, isp, etc., the preset data structure may be set as IP address-province-city-Isp-other remark data.
And step S206, converting the record data of the IP address into a key value pair form and storing the key value pair form in an IP database.
In this embodiment, the record data of each IP address may be converted into a key value pair form and stored in the IP database. The Key Value pair form is Key-Value, and when the Key Value pair form is stored, the record data of the IP address is stored according to the Key Value pair form, which can also be called KV storage, wherein the IP address can be stored as Key, and the analysis result corresponding to the IP address can be stored as Value.
In a specific implementation, the IP database may be deployed in the Redis database system, so that KV attribute of the Redis database may be utilized, all keys are IP addresses, and when an IP address resolution request is received each time, a resolution result corresponding to the IP address is returned in real time by querying a local cache of the Redis database system and the Redis database.
Several alternative embodiments are provided below to optimize the method of building an IP database, in particular as follows:
in a preferred embodiment of the present application, the step S202 may include the steps of:
and obtaining analysis results corresponding to all the IP addresses from the IP library server according to a preset period.
In this embodiment, the analysis results corresponding to all the IP addresses may be obtained from the IP library server through the IP data crawler service according to a preset period. The preset period may be a preset timing period, and if the preset period is 24 hours, the analysis results corresponding to all the IP addresses are obtained from the IP library server through the IP data crawler service every 24 hours, so as to update the record data of the IP addresses stored in the IP database.
In a preferred embodiment of the present application, after said step S204, the method further comprises:
uploading the recorded data of the IP address to a file management system; and reading the record data of the IP address from the file management system through an IP data storage system, and storing the record data of the IP address.
Specifically, the file management system may be deployed in a remote server, where the IP data crawler service generates the record data of the IP addresses, then stores the record data of the IP addresses in a local file system, and uploads the record data of the IP addresses to the remote file management system after the daily crawler update data is completed. And then the recorded data of the IP address is read from the file management system through the IP data storage system, and the recorded data of the IP address is stored.
The IP data storage system also transmits the record data of the updated IP address in the period to the file management system according to the preset period by the fixed equal IP data crawler service, and then downloads the record data of the latest IP address to the local and stores the record data of the IP addresses.
In a preferred embodiment of the present application, the IP database is deployed in a Redis database system, and the step S206 may include the steps of:
and reading the record data of the IP address from the IP data storage system through the Redis database system, converting the record data of the IP address into a key value pair form and storing the key value pair form in the IP database.
In this embodiment, after the IP data storage system updates the record data of the latest IP address, the record data of the IP address may be read from the IP data storage system through the Redis database system, and the record data of the IP address may be converted into a Key Value pair form and stored in the IP database, where, when storing, the IP address may be stored as a Key, and the analysis result corresponding to the IP address may be stored as a Value.
In a preferred embodiment of the present application, the method further comprises:
receiving an IP address resolution request; wherein, the IP address analysis request carries the IP address to be analyzed; and inquiring a target analysis result corresponding to the IP address to be analyzed from an IP database.
Specifically, an IP address resolution request sent by a user may be received through an IP database query system, where the IP address resolution request carries an IP address to be resolved. After receiving the IP address resolution request, extracting an IP address to be resolved from the IP address resolution request, and then inquiring a target resolution result corresponding to the IP address to be resolved from an IP database.
In a preferred embodiment of the present application, the querying, from an IP database, a target resolution result corresponding to the IP address to be resolved includes:
inquiring from the Key of an IP database to obtain the IP address to be resolved, and determining a target resolving result corresponding to the IP address to be resolved from the Value of the IP database.
In this embodiment, since the record data of the IP address is converted into a Key Value pair form in advance and stored in the IP database, the IP address is stored as a Key when storing, and the analysis result corresponding to the IP address is stored as a Value. Therefore, the IP database query system can directly query from the Key of the IP database to obtain the IP address to be resolved, and determine the target resolving result corresponding to the IP address to be resolved from the Value of the IP database.
In a preferred embodiment of the present application, the method further comprises:
and accessing the server corresponding to the IP address to be resolved according to the target resolving result.
In this embodiment, after the IP database query system queries to obtain the target resolution result corresponding to the IP address to be resolved, the IP database query system may access the server corresponding to the IP address to be resolved according to the target resolution result, so as to obtain the data resource in the server corresponding to the IP address to be resolved.
In a preferred embodiment of the present application, the method further comprises:
forwarding the IP address resolution request to an IP library server under the condition that the access to the server corresponding to the IP address to be resolved fails according to the target resolution result; receiving a real-time analysis result corresponding to the IP address to be analyzed, which is returned by the IP library server; and accessing a server corresponding to the IP address to be resolved according to the real-time resolving result.
In this embodiment, when accessing the server corresponding to the IP address to be resolved according to the target resolution result fails, the IP database query system needs to acquire the real-time resolution result corresponding to the IP address to be resolved from the IP database server again. Specifically, an IP address analysis request is forwarded to an IP library server, then a real-time analysis result corresponding to an IP address to be analyzed, which is returned by the IP library server, is received, and the server corresponding to the IP address to be analyzed is accessed according to the real-time analysis result, so that data resources in the server corresponding to the IP address to be analyzed are obtained.
To further describe the method for constructing an IP database in this embodiment, as shown in fig. 4, a modular block diagram of a method for constructing an IPR database includes an IP library server, an IP data crawler service, a file management system, an IP data storage system, an IP database (redis) and an IP database query system, and the process of processing data by each module is specifically as follows:
IP data crawler service: the IP data crawler service obtains analysis results corresponding to all IP addresses from an IP library server in a traversing way at regular time according to a preset period, wherein the IP library server can be an HTTP server of an IP library server provider and is used for providing IP address analysis service for vast Internet users, and data obtained by the IP data crawler service from the IP library server is public and licensed legal data. Meanwhile, after the IP data crawler service crawls and obtains analysis results corresponding to all the IP addresses, each result is recorded as a row according to a preset data structure and is stored in a local file system of the IP data crawler service.
A file management system: after the IP data crawler service captures the analysis result data corresponding to all the IP addresses, the data are uploaded to a remote file management system.
IP data storage system: the IP data storage system transmits the analysis result data corresponding to the updated IP address to the file management system according to the preset period, periodically and the like, and then downloads the analysis result data corresponding to the latest IP address to the local and stores the data.
IP database (redis): and analyzing each row of records in the file according to the agreed data format by utilizing the analysis result corresponding to the IP address stored in the IP data storage system, and converting the records into kv information in the IP database at the same time, thereby realizing the data conversion from the file management system to the redis database.
IP database query system: the KV attribute of the Redis database is utilized, all keys are IP addresses, and when an IP address analysis request is received each time, the analysis result corresponding to the IP address is determined by inquiring the data returned by the IP database in real time. In addition, the IP database query system may further set a local cache system, when receiving the IP address resolution request, first query whether there is a resolution result corresponding to the IP address from the local cache, and if not, query the resolution result corresponding to the IP address from the IP database.
In this embodiment, the IP data crawler service obtains the analysis results corresponding to all the IP addresses offline by using the manner of deploying the IP database offline, and then uses the file management system and the Redis database to transfer and store the analysis result data corresponding to all the IP addresses, so as to achieve the purpose of constructing the offline IP database, thereby directly querying the analysis results corresponding to the IP addresses from the IP database when the IP addresses need to be analyzed, achieving the purpose of reducing the IP analysis delay, and improving the access capability of the IP data analysis service.
Example two
Fig. 4 schematically shows a block diagram of an apparatus for constructing an IP database according to the second embodiment of the present application, which may be divided into one or more program modules, which are stored in a storage medium and executed by one or more processors to complete the embodiments of the present application. Program modules in the embodiments of the present application refer to a series of computer program instruction segments capable of implementing specific functions, and the following description specifically describes the functions of each program module in the embodiments of the present application.
As shown in fig. 4, the apparatus 400 for constructing an IP database may include the following modules:
the data obtaining module 410 is configured to obtain analysis results corresponding to all IP addresses from the IP library server;
the data conversion module 420 is configured to convert, according to a preset data structure, the analysis result corresponding to each IP address to obtain record data of the IP address;
the data storage module 430 is configured to convert the record data of the IP address into a key value pair and store the key value pair in the IP database.
In a preferred embodiment of the present application, the data acquisition module 410 includes:
and the data acquisition sub-module is used for acquiring analysis results corresponding to all the IP addresses from the IP library server according to a preset period.
In a preferred embodiment of the present application, the apparatus further comprises:
the data uploading module is used for uploading the recorded data of the IP address to a file management system;
and the data storage module is used for reading the record data of the IP address from the file management system through the IP data storage system and storing the record data of the IP address.
In a preferred embodiment of the present application, the IP database is deployed in a Redis database system, and the data storage module 430 includes:
and the data storage sub-module is used for reading the record data of the IP address from the IP data storage system through the Redis database system, converting the record data of the IP address into a key value pair form and storing the key value pair form in the IP database.
In a preferred embodiment of the present application, the apparatus further comprises:
the analysis request receiving module is used for receiving the IP address analysis request; wherein, the IP address analysis request carries the IP address to be analyzed;
and the data query module is used for querying a target analysis result corresponding to the IP address to be analyzed from an IP database.
In a preferred embodiment of the present application, the data query module includes:
and the data query sub-module is used for querying from the Key of the IP database to obtain the IP address to be resolved, and determining a target resolving result corresponding to the IP address to be resolved from the Value of the IP database.
In a preferred embodiment of the present application, the apparatus further comprises:
and the first data access module is used for accessing the server corresponding to the IP address to be resolved according to the target resolution result.
In a preferred embodiment of the present application, the apparatus further comprises:
the request forwarding module is used for forwarding the IP address analysis request to an IP library server under the condition that the server corresponding to the IP address to be analyzed is failed to be accessed according to the target analysis result;
the analysis result receiving module is used for receiving a real-time analysis result corresponding to the IP address to be analyzed, which is returned by the IP library server;
and the second data access module is used for accessing the server corresponding to the IP address to be resolved according to the real-time resolution result.
Example III
Fig. 5 schematically shows a hardware architecture diagram of a computer device 10000 suitable for implementing a method of building an IP database according to the third embodiment of the present application. In this embodiment, the computer device 10000 is a device capable of automatically performing numerical calculation and/or information processing in accordance with an instruction set or stored in advance. For example, the server may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a rack-mounted server, a blade server, a tower server, or a cabinet server (including a FEN independent server or a server cluster formed by a plurality of servers), etc. As shown in fig. 5, computer device 10000 includes at least, but is not limited to: the memory 10010, processor 10020, network interface 10030 may be communicatively linked to each other via a system bus. Wherein:
memory 10010 includes at least one type of computer-readable storage medium including flash memory, hard disk, multimedia card, card memory (e.g., SD or DX memory, etc.), random Access Memory (RAM), static Random Access Memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, memory 10010 may be an internal storage module of computer device 10000, such as a hard disk or memory of computer device 10000. In other embodiments, the memory 10010 may also be an external storage device of the computer device 10000, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card) or the like. Of course, the memory 10010 may also include both an internal memory module of the computer device 10000 and an external memory device thereof. In this embodiment, the memory 10010 is typically used for storing an operating system installed on the computer device 10000 and various types of application software, such as program codes of a method of constructing an IP database. In addition, the memory 10010 may be used to temporarily store various types of data that have been output or are to be output.
The processor 10020 may be a central processing unit (Central Processing Unit, simply CPU), controller, microcontroller, microprocessor, or other data processing chip in some embodiments. The processor 10020 is typically configured to control overall operation of the computer device 10000, such as performing control and processing related to data interaction or communication with the computer device 10000. In this embodiment, the processor 10020 is configured to execute program codes or process data stored in the memory 10010.
The network interface 10030 may comprise a wireless network interface or a wired network interface, which network interface 10030 is typically used to establish a communication link between the computer device 10000 and other computer devices. For example, the network interface 10030 is used to connect the computer device 10000 to an external terminal through a network, establish a data transmission channel and a communication link between the computer device 10000 and the external terminal, and the like. The network may be a wireless or wired network such as an Intranet (Intranet), the Internet (Internet), a global system for mobile communications (Global System of Mobile communication, abbreviated as GSM), wideband code division multiple access (Wideband Code Division Multiple Access, abbreviated as WCDMA), a 4G network, a 5G network, bluetooth (Bluetooth), wi-Fi, etc.
It should be noted that fig. 5 only shows a computer device having components 10010-10030, but it should be understood that not all of the illustrated components are required to be implemented, and more or fewer components may be implemented instead.
In this embodiment, the method for constructing an IP database stored in the memory 10010 may be further divided into one or more program modules and executed by one or more processors (the processor 10020 in this embodiment) to complete the embodiments of the present application.
Example IV
The present application also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the method of constructing an IP database in the embodiments.
In this embodiment, the computer-readable storage medium includes a flash memory, a hard disk, a multimedia card, a card memory (e.g., SD or DX memory, etc.), a Random Access Memory (RAM), a Static Random Access Memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, an optical disk, and the like. In some embodiments, the computer readable storage medium may be an internal storage unit of a computer device, such as a hard disk or a memory of the computer device. In other embodiments, the computer readable storage medium may also be an external storage device of a computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash Card (Flash Card), etc. that are provided on the computer device. Of course, the computer-readable storage medium may also include both internal storage units of a computer device and external storage devices. In this embodiment, the computer readable storage medium is typically used to store an operating system installed on a computer device and various types of application software, such as program codes of the method of constructing an IP database in the embodiment, and the like. Furthermore, the computer-readable storage medium may also be used to temporarily store various types of data that have been output or are to be output.
It will be apparent to those skilled in the art that the modules or steps of the embodiments of the application described above may be implemented in a general purpose computing device, they may be concentrated on a single computing device, or distributed across a network of computing devices, they may alternatively be implemented in program code executable by computing devices, so that they may be stored in a storage device for execution by computing devices, and in some cases, the steps shown or described may be performed in a different order than what is shown or described, or they may be separately fabricated into individual integrated circuit modules, or multiple modules or steps of them may be fabricated into a single integrated circuit module. Thus, embodiments of the present application are not limited to any specific combination of hardware and software.
The foregoing description is only of the preferred embodiments of the present application, and is not intended to limit the scope of the claims, and all equivalent structures or equivalent processes using the descriptions and drawings of the present application, or direct or indirect application in other related technical fields are included in the scope of the claims of the present application.
Claims (11)
1. A method of constructing an IP database, the method comprising:
the analysis results corresponding to all the IP addresses are obtained from the IP library server;
aiming at the analysis result corresponding to each IP address, converting according to a preset data structure to obtain the record data of the IP address;
and converting the recorded data of the IP address into a key value pair form and storing the key value pair form in an IP database.
2. The method for constructing an IP database according to claim 1, wherein the obtaining, from the IP library server, the resolution results corresponding to all the IP addresses includes:
and obtaining analysis results corresponding to all the IP addresses from the IP library server according to a preset period.
3. The method for constructing an IP database according to claim 1, wherein after the step of converting the parsing result corresponding to each IP address according to a preset data structure to obtain the record data of the IP address, the method further comprises:
uploading the recorded data of the IP address to a file management system;
and reading the record data of the IP address from the file management system through an IP data storage system, and storing the record data of the IP address.
4. A method of constructing an IP database according to claim 3, wherein the IP database is deployed in a Redis database system, and the converting the record data of the IP address into a key-value pair form is stored in the IP database, comprising:
and reading the record data of the IP address from the IP data storage system through the Redis database system, converting the record data of the IP address into a key value pair form and storing the key value pair form in the IP database.
5. The method of constructing an IP database of claim 1, further comprising:
receiving an IP address resolution request; wherein, the IP address analysis request carries the IP address to be analyzed;
and inquiring a target analysis result corresponding to the IP address to be analyzed from an IP database.
6. The method for constructing an IP database according to claim 5, wherein said querying, from the IP database, a target resolution result corresponding to the IP address to be resolved, includes:
inquiring from the Key of an IP database to obtain the IP address to be resolved, and determining a target resolving result corresponding to the IP address to be resolved from the Value of the IP database.
7. The method of constructing an IP database of claim 5, further comprising:
and accessing a server corresponding to the IP address to be resolved according to the target resolution result.
8. The method of constructing an IP database of claim 7, further comprising:
forwarding the IP address resolution request to an IP library server under the condition that the access to the server corresponding to the IP address to be resolved fails according to the target resolution result;
receiving a real-time analysis result corresponding to the IP address to be analyzed, which is returned by the IP library server;
and accessing a server corresponding to the IP address to be resolved according to the real-time resolving result.
9. An apparatus for building an IP database, the apparatus comprising:
the data acquisition module is used for acquiring analysis results corresponding to all the IP addresses from the IP library server;
the data conversion module is used for converting the analysis result corresponding to each IP address according to a preset data structure to obtain the record data of the IP address;
and the data storage module is used for converting the record data of the IP address into a key value pair form and storing the key value pair form in an IP database.
10. A computer device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor is adapted to carry out the steps of the method of constructing an IP database according to any one of claims 1 to 8 when the computer program is executed by the processor.
11. A computer-readable storage medium, in which a computer program is stored, the computer program being executable by at least one processor to cause the at least one processor to perform the steps of the method of constructing an IP database as claimed in any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311863945.1A CN117807344A (en) | 2023-12-29 | 2023-12-29 | Method and device for constructing IP database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311863945.1A CN117807344A (en) | 2023-12-29 | 2023-12-29 | Method and device for constructing IP database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117807344A true CN117807344A (en) | 2024-04-02 |
Family
ID=90419871
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311863945.1A Pending CN117807344A (en) | 2023-12-29 | 2023-12-29 | Method and device for constructing IP database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117807344A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119577054A (en) * | 2025-02-07 | 2025-03-07 | 广东南方智媒科技有限公司 | High-performance IP-to-geolocation method and system based on IP offline library |
-
2023
- 2023-12-29 CN CN202311863945.1A patent/CN117807344A/en active Pending
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119577054A (en) * | 2025-02-07 | 2025-03-07 | 广东南方智媒科技有限公司 | High-performance IP-to-geolocation method and system based on IP offline library |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11836533B2 (en) | Automated reconfiguration of real time data stream processing | |
| US11853779B2 (en) | System and method for distributed security forensics | |
| US10447772B2 (en) | Managed function execution for processing data streams in real time | |
| US10129118B1 (en) | Real time anomaly detection for data streams | |
| CN111182089B (en) | Container cluster system, method and device for accessing big data assembly and server | |
| US20170185678A1 (en) | Crawler system and method | |
| CN112416964A (en) | Data processing method, device and system, computer equipment and computer readable storage medium | |
| US11188443B2 (en) | Method, apparatus and system for processing log data | |
| CN109151025B (en) | Load balancing method and device based on URL, computer storage medium and equipment | |
| CN117389830A (en) | Cluster log acquisition method and device, computer equipment and storage medium | |
| CN117807344A (en) | Method and device for constructing IP database | |
| CN114297559B (en) | Dynamic interactive loading optimization method for web programs and U3D applications | |
| CN113360558B (en) | Data processing method, data processing device, electronic equipment and storage medium | |
| CN119557055A (en) | Cross-Kubernetes cluster service access method, device, computer equipment and storage medium | |
| CN117082029B (en) | Access method and device based on line cache, electronic equipment and storage medium | |
| US12132804B2 (en) | Runtime module conversion | |
| CN113612811B (en) | Method, system, equipment and medium for client mounting in multiple channels | |
| CN114124883B (en) | Data access method and device based on cloud storage address, computer equipment and medium | |
| WO2023143061A1 (en) | Data access method and data access system thereof | |
| CN117119227A (en) | Page information acquisition method and device, computer equipment and storage medium | |
| US10402391B2 (en) | Processing method, device and system for data of distributed storage system | |
| CN113760836A (en) | Wide table calculation method and device | |
| US12547314B2 (en) | Configurable and event driven smart hybrid cloud orchestrator | |
| CN116126217B (en) | Storage resource allocation method, device, equipment and storage medium | |
| CN116305288B (en) | Method, device, equipment and storage medium for isolating database resources |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |