Unique versus Raw

sarettah · 05-19-2005

Just a note after reading a different thread.

Those that know about counting dynamic ips etc need read no further, those (especially programmers that write the stuff that counts unique hits and don't know this shit, you can continue to read)

Back in the day, when I was working for ODC, one of my original primary responsibilities was writing the code to count the raw versus unique counts each day. We took painstaking methods to get those numbers as accurate as possible,

A difficulty any program can have in the computation is dynamic ips, where an isp uses the same ip over and over and over again for different users each time.

Now, if you sre setting cookies, then that is going to give you the most accurate count (imho), but, if you are trying to compute from raw logs then you have to be aware of which isp it is coming from and whether they use dynamic or statis ips (or server side caching which is another whole kettle of fish)

If you have an isp that uses dynamic caching, especially if it is a larger isp (lets say AOL or roadrunner) you must go a lot deeper than just looking at the ip to determine if it is a new user (unique) or someone who has been there before.

Some of the methodologies we used were:

1. counting the timeouts between hits. If it was within a certain time period (I think we started with 15 minutes and then tuned from there, it was a while ago and my brain aint what it once was) ie: if the same ip hit within a 15 minute time period, it was the same surfer. If over 15 minutes had gone by, then it was a new user.

2. looking at what was coming in as the 'remote_host' string. Quite often an isp will pass member info in the remote string (or the server picks it up on a reverse ip lookup at the time of the hit, however it gets there, it is oftem there) Unique member info in the host string tells you which surfer it is.

3. Looking at the referer string. If the referer was coming from our site(s), then it definitely was NOT a new hit, as the referring string should have been anything other than our sites on a new hit.

There were several other things we looked at, like I said, it was a while ago.

The main thing I am trying (in such a long winded manner) to say is that if you are calling a hit unique or not based solely on the ip address, then you are automatically shaving off a lot of your traffic especially if you have a lot of AOL, Roadrunner, SWBell, SBCGlobal or any of the other large isps traffic.

It is important to know how many actual uniques you are getting so that when you do the shave, you are shaving accurately and not cutting the balls off of your affiliates by accident.

Trev · 05-19-2005

Fuck!!!

I'm shot and out of plain expletives, like damn....

Servers pick up so much info that it wouldn't be hard to check the ip against:

Browser, platform, screen res and added browser extensions (alexa shows up).

But that would make it so much harder to be bad

BillPMB · 05-19-2005

mac address, unique php session ids, stuff like that works good too.

sarettah · 05-19-2005

Quote:

Originally posted by BillPMB@May 19 2005, 09:29 PM
mac address, unique php session ids, stuff like that works good too.

Yeppir. All sorts of good info to come off of. But, first you gotta wanna, ya know

twan · 05-20-2005

Quote:

Originally posted by BillPMB@May 19 2005, 06:29 PM
mac address, unique php session ids, stuff like that works good too.

I always thought a mac address was a number of the network card used.
But when that is true how can i read it in a browser?

BillPMB · 05-20-2005

Quote:

Originally posted by twan@May 20 2005, 04:25 AM
I always thought a mac address was a number of the network card used.
But when that is true how can i read it in a browser?

With one of these thingys!

http://www.networkgeneral.com/Products_det...=20046243936754

Might be a bit clunky, but they work great!

Jeremy · 05-20-2005

Sarettah: UVOVUN

Bill: Glad I read on before posting.... are MAC addresses completely unique?

twan · 05-20-2005

Quote:

« Previous Thread | Next Thread »

Originally posted by BillPMB+May 20 2005, 05:13 AM-->
QUOTE (BillPMB @ May 20 2005, 05:13 AM)

twan

View Public Profile

Find More Posts by twan

05-20-2005 #9

BillPMB

Members

Join Date: Dec 2002

Posts: 308

Each ethernet card has a unique mac address. The IP address associated with that mac address is the one closest to that address, hence the source IP address.

All addresses subsequent to the 1st address are network hops between source and destination.

Clear as mud?

__________________
Why the hell would we shave? We make money when you do.

BillPMB

View Public Profile

Find More Posts by BillPMB

05-20-2005 #10

sarettah

Stay Out Of The Box

Want to see your own Advertising Here!

Join Date: Oct 2002

Location: Lost in the ozone

Posts: 10,318

Quote:

Originally posted by Jeremy@May 20 2005, 10:44 AM
Sarettah: UVOVUN

__________________

Free Adult Webcams

sarettah

View Public Profile

Send a private message to sarettah

Find More Posts by sarettah

Bookmarks

Google

Thread Tools

Show Printable Version

Email this Page

Posting Rules
You may not post new threads You may not post replies You may not post attachments You may not edit your posts BB code is On Smilies are On [IMG] code is On HTML code is On

All times are GMT -4. The time now is 07:56 PM..

05-19-2005	#2
Trev Cunty Want to see your own Advertising Here! Join Date: Jan 2003 Posts: 10,150	Fuck!!! I'm shot and out of plain expletives, like damn.... Servers pick up so much info that it wouldn't be hard to check the ip against: Browser, platform, screen res and added browser extensions (alexa shows up). But that would make it so much harder to be bad __________________ Sucks to be you.

05-19-2005	#1
sarettah Stay Out Of The Box Want to see your own Advertising Here! Join Date: Oct 2002 Location: Lost in the ozone Posts: 10,318	Just a note after reading a different thread. Those that know about counting dynamic ips etc need read no further, those (especially programmers that write the stuff that counts unique hits and don't know this shit, you can continue to read) Back in the day, when I was working for ODC, one of my original primary responsibilities was writing the code to count the raw versus unique counts each day. We took painstaking methods to get those numbers as accurate as possible, A difficulty any program can have in the computation is dynamic ips, where an isp uses the same ip over and over and over again for different users each time. Now, if you sre setting cookies, then that is going to give you the most accurate count (imho), but, if you are trying to compute from raw logs then you have to be aware of which isp it is coming from and whether they use dynamic or statis ips (or server side caching which is another whole kettle of fish) If you have an isp that uses dynamic caching, especially if it is a larger isp (lets say AOL or roadrunner) you must go a lot deeper than just looking at the ip to determine if it is a new user (unique) or someone who has been there before. Some of the methodologies we used were: 1. counting the timeouts between hits. If it was within a certain time period (I think we started with 15 minutes and then tuned from there, it was a while ago and my brain aint what it once was) ie: if the same ip hit within a 15 minute time period, it was the same surfer. If over 15 minutes had gone by, then it was a new user. 2. looking at what was coming in as the 'remote_host' string. Quite often an isp will pass member info in the remote string (or the server picks it up on a reverse ip lookup at the time of the hit, however it gets there, it is oftem there) Unique member info in the host string tells you which surfer it is. 3. Looking at the referer string. If the referer was coming from our site(s), then it definitely was NOT a new hit, as the referring string should have been anything other than our sites on a new hit. There were several other things we looked at, like I said, it was a while ago. The main thing I am trying (in such a long winded manner) to say is that if you are calling a hit unique or not based solely on the ip address, then you are automatically shaving off a lot of your traffic especially if you have a lot of AOL, Roadrunner, SWBell, SBCGlobal or any of the other large isps traffic. It is important to know how many actual uniques you are getting so that when you do the shave, you are shaving accurately and not cutting the balls off of your affiliates by accident. __________________ Free Adult Webcams

05-19-2005	#3
BillPMB Members Join Date: Dec 2002 Posts: 308	mac address, unique php session ids, stuff like that works good too. __________________ Why the hell would we shave? We make money when you do.

05-20-2005	#7
Jeremy Members Join Date: Apr 2005 Posts: 157	Sarettah: UVOVUN Bill: Glad I read on before posting.... are MAC addresses completely unique?