PDA

View Full Version : Unique versus Raw


sarettah
05-19-2005, 07:52 PM
Just a note after reading a different thread.

Those that know about counting dynamic ips etc need read no further, those (especially programmers that write the stuff that counts unique hits and don't know this shit, you can continue to read)

Back in the day, when I was working for ODC, one of my original primary responsibilities was writing the code to count the raw versus unique counts each day. We took painstaking methods to get those numbers as accurate as possible,

A difficulty any program can have in the computation is dynamic ips, where an isp uses the same ip over and over and over again for different users each time.

Now, if you sre setting cookies, then that is going to give you the most accurate count (imho), but, if you are trying to compute from raw logs then you have to be aware of which isp it is coming from and whether they use dynamic or statis ips (or server side caching which is another whole kettle of fish)

If you have an isp that uses dynamic caching, especially if it is a larger isp (lets say AOL or roadrunner) you must go a lot deeper than just looking at the ip to determine if it is a new user (unique) or someone who has been there before.

Some of the methodologies we used were:

1. counting the timeouts between hits. If it was within a certain time period (I think we started with 15 minutes and then tuned from there, it was a while ago and my brain aint what it once was) ie: if the same ip hit within a 15 minute time period, it was the same surfer. If over 15 minutes had gone by, then it was a new user.

2. looking at what was coming in as the 'remote_host' string. Quite often an isp will pass member info in the remote string (or the server picks it up on a reverse ip lookup at the time of the hit, however it gets there, it is oftem there) Unique member info in the host string tells you which surfer it is.

3. Looking at the referer string. If the referer was coming from our site(s), then it definitely was NOT a new hit, as the referring string should have been anything other than our sites on a new hit.

There were several other things we looked at, like I said, it was a while ago.

The main thing I am trying (in such a long winded manner) to say is that if you are calling a hit unique or not based solely on the ip address, then you are automatically shaving off a lot of your traffic especially if you have a lot of AOL, Roadrunner, SWBell, SBCGlobal or any of the other large isps traffic.

It is important to know how many actual uniques you are getting so that when you do the shave, you are shaving accurately and not cutting the balls off of your affiliates by accident. :yowsa:

Trev
05-19-2005, 08:05 PM
Fuck!!!


I'm shot and out of plain expletives, like damn....

Servers pick up so much info that it wouldn't be hard to check the ip against:

Browser, platform, screen res and added browser extensions (alexa shows up).




But that would make it so much harder to be bad :redance:

BillPMB
05-19-2005, 10:28 PM
mac address, unique php session ids, stuff like that works good too.

:)

sarettah
05-19-2005, 11:09 PM
Originally posted by BillPMB@May 19 2005, 09:29 PM
mac address, unique php session ids, stuff like that works good too.

:)
Yeppir. All sorts of good info to come off of. But, first you gotta wanna, ya know :)

twan
05-20-2005, 08:24 AM
Originally posted by BillPMB@May 19 2005, 06:29 PM
mac address, unique php session ids, stuff like that works good too.

:)
I always thought a mac address was a number of the network card used.
But when that is true how can i read it in a browser?

BillPMB
05-20-2005, 09:12 AM
Originally posted by twan@May 20 2005, 04:25 AM
I always thought a mac address was a number of the network card used.
But when that is true how can i read it in a browser?
With one of these thingys!

http://www.networkgeneral.com/Products_det...=20046243936754 (http://www.networkgeneral.com/Products_details.aspx?PrdId=20046243936754)

Might be a bit clunky, but they work great!

:yowsa:

Jeremy
05-20-2005, 11:43 AM
Sarettah: UVOVUN

Bill: Glad I read on before posting.... are MAC addresses completely unique?

twan
05-20-2005, 11:44 AM
Originally posted by BillPMB+May 20 2005, 05:13 AM--></div><table border='0' align='center' width='95%' cellpadding='3' cellspacing='1'><tr><td>QUOTE (BillPMB @ May 20 2005, 05:13 AM)</td></tr><tr><td id='QUOTE'> <!--QuoteBegin-twan@May 20 2005, 04:25 AM
I always thought a mac address was a number of the network card used.
But when that is true how can i read it in a browser?
With one of these thingys!

http://www.networkgeneral.com/Products_det...=20046243936754 (http://www.networkgeneral.com/Products_details.aspx?PrdId=20046243936754)

Might be a bit clunky, but they work great!

:yowsa: [/b][/quote]
Getting a bit technical below :)
If i understand correctly the page you mentioned offers a sniffer. A sniffer can indeed take the mac address of a ip packet.
But the thing is that the mac address changes whenever the ip packet is send through a router. So you most likely will end up with the mac address of the last router (in the route from the client to your server). This mac address will likely be the mac address of the router used by your hoster.
I'm not really sure how the mac address of your hoster router could be usefull when defining if a visitor of your website is unique. Can someone enlighten me?

BillPMB
05-20-2005, 12:50 PM
Each ethernet card has a unique mac address. The IP address associated with that mac address is the one closest to that address, hence the source IP address.

All addresses subsequent to the 1st address are network hops between source and destination.

Clear as mud?

sarettah
05-20-2005, 01:18 PM
Originally posted by Jeremy@May 20 2005, 10:44 AM
Sarettah: UVOVUN

:blink: