Hi,
I have a file with a list of IP addresses(about 23 millions records) and I would like to put respective countries for the IPs. For that I have another file which has a range of IP addresses (in Integer form) and I came up with the attached KTR which is running very slow. I was wondering if there is a way to look up a range.
Layout of IpToCountry file -
startIpNum,endIpNum,NotInUse1,NotInUse2,CountryCode2Dig,CountryCode3Dig,Country Name
In the KTR, I first calculate the respective IP integer for an IP address (say IPnum). If this IPnum is between startIpNum and endIpNum above then take that country name using "CartesianJoin", which I think is the bottle neck.
Please suggest a better way to lookup range.
Much appreciated.
UPDATE: Out of 23 millions, there are only 4 million unique IP addresses. I'll try if looking up for unique IPs improves the performance to acceptable.
I have a file with a list of IP addresses(about 23 millions records) and I would like to put respective countries for the IPs. For that I have another file which has a range of IP addresses (in Integer form) and I came up with the attached KTR which is running very slow. I was wondering if there is a way to look up a range.
Layout of IpToCountry file -
startIpNum,endIpNum,NotInUse1,NotInUse2,CountryCode2Dig,CountryCode3Dig,Country Name
In the KTR, I first calculate the respective IP integer for an IP address (say IPnum). If this IPnum is between startIpNum and endIpNum above then take that country name using "CartesianJoin", which I think is the bottle neck.
Please suggest a better way to lookup range.
Much appreciated.
UPDATE: Out of 23 millions, there are only 4 million unique IP addresses. I'll try if looking up for unique IPs improves the performance to acceptable.