Improved unique hits data

When I first created Tinylytics it was important to allow support for collecting unique hits/views. This was achieved by passing along a randomly generated set of letters and numbers (token) to the browser when the embed script loaded if you had unique hits enabled in your site options.

That allowed it to tie the user session to that specific browser without knowing anything about the user. When the user then navigated around your site, it would see this random token within the temporary browser session storage and pass it back when creating a hit — this worked well. With that data, it was easy to correlate hits that contained the same token that was passed back.

This week brings a change on how this works and I believe will be even better than the original system, whilst fully preserving privacy.

From today onward, irrespective of your settings, Tinylytics will automatically generate a unique identifier for each hit recorded for your site.

How it works

You might be asking how this works without compromising privacy of your readers/users, however I have you covered.

Instead of passing back a random token as the script loads, a unique hash is generated as the hit/view is processed by Tinylytics.

When a hit is recorded, the script passes back your usual data, like the URL, your site id, any parameters that might be useful (ref, utm_source etc) to the collector. In addition to this, the collector will get the browser (user agent) the request was made from, including the connecting IP address from Cloudflare.

It's important to note that only the connecting Cloudflare IP is sent to the server. The real IP of the user will never reach our service and is not available to use. Cloudflare scrubs this before the request is sent to the server. That's on purpose and I wouldn't have it any other way. In essence, when the collector endpoint is called, it will be routed to the nearest Cloudflare server (this is the IP we get) to the user, which in turn then sends the request onward to the Tinylytics server.

Using all this data we then create a unique hash combining all this data into one. The unique hash is combined with a rotated salt (a randomly generated secret) so that decryption is not possible. Salt keys are automatically rotated every 12 hours and are deleted from the system at a specified time window. Salt keys are stored encrypted and are not stored in plain text.

All of this allows us to more easily track unique views across your site and URLs. Because the incoming data should be the same, unless the Cloudflare IP changes, it's a great way, and much better one, to tie each hit to a unique user... without ever knowing anything about them that invades their privacy.

As always, we do store and process the user agent for the purpose of sorting hits by browser and platform. The full user agent string is stored for 7 days and then is removed automatically after the necessary data has been populated. This allows a small window for debugging purposes.

With the 12 hour window, it will also give a better view of returning visitors within that same time frame, instead of sessions by session. Session storage on browsers doesn't always work as intended and might be different depending on the tab they have open, or browser they use (because some browsers just clear it or refuse to save this token to session storage). That could cause weird or mostly inaccurate data.

On the plus side, it's a little less script to load now too!

The new default

Unique hit data will always be recorded irrespective if you're on the paid plan or not. If you're on the free plan, and upgrade at a later stage, you'll automatically see any past unique data (starting from today).

As I explained, it's the new default because all we're using is the exact same data that we previously had with each request — so it doesn't make a difference. In addition to the salts, as a precaution, it makes this robust and privacy preserving.

I did try my hand at explaining how this all works a little bit more here in the documentation. The docs need a little bit more polish though.

Anyway, hope you like this and please let me know any oddities.