What’s in a session: What exactly happens during a session, and how does the xDB know who you are?

This blog post follows on from Let’s Talk About Session State. Now that we understand the role played by session state in the xDB, let’s take a closer look at what exactly happens during a session – and how xDB is able to keep track of us across sessions.

But first – the setup!

I am using OutProc session state (specifically the MongoDB provider) for private and shared so that I can see what is happening to my session. They are using the same database, which is possible as long as you make sure that the sessionType properties on the provider differ – e.g. sessionType="private".

I also created this handy-dandy rendering that calls  It outputs the ASP.NET cookie, the analytics cookie, and the currently identified contact. The text box just calls Tracker.Current.Session.Identify().

It also has quick links for abandoning session and clearing cookies in various combinations:


What happens when you identify a contact for the first time?

I come to Sitecore as an anonymous user – Sitecore issues me with an analytics cookie and an ASP.NET cookie. If I look in the session database, I can see that I have an entry for shared session state (which uses the same ID as my analytics cookie) and an entry for private session state (which uses the same ID as my ASP.NET cookie):


As soon as I identify myself, I immediately get an entry in the xDB – notice that my ID matches the analytics cookie. This is because it’s my first visit, and the first time that I have identified myself:


I did something at the same time as identifying my contact. I set a number of facets:


However, these have not yet appeared in the collection database. Unless you force data to be flushed to the xDB (more on this later), it will not appear in the collection database until the session ends.

What happens when the session ends, and how can I force the session to end?

By default, sessions time out after 20 minutes of inactivity. Private session state times out first, and shared session state times out 1 minute later. You can modify the timeout setting in web.config – search for the sessionState node and change the timeout property.

If you want to force a session to time out for testing purposes, you can call System.Web.HttpContext.Current.Session.Abandon().

From this page, I linked to a page that abandons the session – in my case, http://mysite.local/abandon

As part of abandoning the session, I also redirected to a page that doesn’t exist. You don’t have to do this, but for debugging purposes I did not want a brand new session to be created as soon as I hit a new page.  As far as I can tell, this does not happen if you redirect to ‘page not found’.

The following screenshot shows what happened to my session. At this point I have not browsed away from the ‘Not Found’ page! This is important, as a new private session with the same ID will be created as soon as I go back to the site

Notice that although private session state disappeared, shared session state is still in there:


If I check the collection database, you can see that my facet data has now appeared – but the contact will have a ‘lease’ on it until the shared session state disappears:


Watch what happens when I visit a page now that I have abandoned my session:


Sitecore still knows who I am. We are stalking you just a lil’ bit.


Why does Sitecore still know who I am, even though I abandoned the session?

Abandoning your session does not delete your cookies. Because your original analytics cookie is present, Sitecore is able to identify you. It will also start a new session with the same ID as the previous one. This is because my original ASP.NET cookie is also still available.

Here is another contact:


And after we abandon session, here is the analytics cookie (still intact):


Here is the ASP.NET cookie (still intact):


Using the same session ID twice does not mean that you will continue the same interaction. It is a brand new interaction using the same session ID.

What happens if I delete my ASP.NET cookie after my first visit?

Sitecore is still able to identify you using the original analytics cookie – you can delete the ASP.NET cookie 3 times and Sitecore will still be able to identify you. Here I have cleared my ASP.NET cookie 3 times;the highlighted entries are private session state:


Sitecore will, however, interpret each fresh ASP.NET cookie as a new visit. I mention this because if you clear your ASP.NET cookie on logout, you are creating two separate visits for that contact.

If I delete my analytics cookie 3 times I must abandon the final session before I see any change to my visit count. My visit count will increment by 3 but only the last active private session actually disappears from the session database immediately; the others are orphaned and left to expire on their own. The expiry time of shared session state is updated to be +1 minute after the final private session.

Here you can see that my visit count is now 3 (keep in mind that you will only see the interactions when the sessions have expired):


Fortunately, contact data is stored in shared session state. This means that whatever I did in my first 19 sessions, the order in which those sessions expire do not affect what contact data is saved to the xDB.

Let’s say that I changed my name in session 3 (Myrtle to Martin), and again in session 10 when I discovered a typo (Martin to Martina). Even if session 3 expires last, the final value of that facet will still be Martina; not Martin.

What happens if I delete my analytics cookie after my first visit?

The ID you see in the analytics cookie on your first visit will become your contact GUID.

If you delete this cookie AND abandon your session (which deletes your private session state), Sitecore has no way of figuring out who you are.

As far as I can tell, the steps are (in no particular order):

  1. Check ASP.NET cookie – does this person have an active session? If yes, they can be identified.
  2. If no, check the analytics cookie – does this cookie match a contact GUID? If yes, they can be identified.
  3. If no – does the the analytics cookie references  a device (see below for explanation) that has a ‘last known contact’ ID? If yes, they can be identified.

My session has expired and I have an analytics cookie that does not match my contact GUID – how does Sitecore still know who I am?

Every time you identify yourself, Sitecore associates your contact GUID with whatever the current analytics cookie ID is. It does that by creating an entry in the devices table. If you clear all cookies and re-identify yourself, Sitecore will create a record like the one below that links the analytics cookie ID to your contact ID:


When this new session expires and you re-visit the site, Sitecore is still able to identify you. Essentially, you have to re-identify yourself each time the analytics cookie is cleared, and a high percentage of non-technical people don’t actually clear their cookies that often (shock horror, I know)!

Forcing data to flush to the xDB

We know that we can force data to flush to the xDB by abandoning our session. You can also force session data to flush without abandoning the session:

I added these lines to my ‘identify me’ form, which means that clicking the ‘Submit’ button creates a contact, populates the facets, and flushes session data at the same time. Here is my contact data (identified as mudkip):


My facet data was available immediately; I did not need to abandon session:


My visit count remained at 1 even after I abandoned the session and cleared all cookies, proving that session data was flushed but the session remained active:


One last thing – mixing InProc and OutProc

I noticed that if I used InProc for private and OutProc for shared (which I do not think there is a reason for; I was just being lazy) my session behaviour became unpredictable – sometimes facets would appear, sometimes not. If anyone else has experienced this, let me know in the comments.

The end

Here is what my list of contact identifiers looked like towards the end of this experiment:


There’s a lot going on with session state. If there is any part of this blog post where your experiences with session state differs, please let me know in the comments or tweet me @mhwelander – I’ll edit in your contributions (with credit, of course) so that we can share the love with the whole community.🙂


Tracker.Current is not initialized

I am writing this for the benefit of anyone else who, like me, goes to Google before checking their logs!

When I switched shared session state to OutProc (using the MongoDB provider), I got the following error:

Tracker.Current is not initialized

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code. 

Exception Details: System.InvalidOperationException: Tracker.Current is not initialized

Source Error: 

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace: 

[InvalidOperationException: Tracker.Current is not initialized]
   Sitecore.Analytics.Pipelines.StartAnalytics.StartTracking.Process(PipelineArgs args) +304
   (Object , Object[] ) +74
   Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +484
   Sitecore.Analytics.Pipelines.StartAnalytics.StartAnalyticsPipeline.Run() +293
   Sitecore.Mvc.Analytics.Pipelines.MvcEvents.RequestBegin.StartTracking.Process(RequestBeginArgs args) +139
   (Object , Object[] ) +74
   Sitecore.Pipelines.CorePipeline.Run(PipelineArgs args) +484
   Sitecore.Mvc.Pipelines.PipelineService.RunPipeline(String pipelineName, TArgs args) +184
   Sitecore.Mvc.Routing.RouteHttpHandler.BeginProcessRequest(HttpContext context, AsyncCallback cb, Object extraData) +74
   System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +921
   System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +137

In my case, it was because I had not marked one of my facets as Serializable:

public class MyFacet : Facet, IMyFacet

Had I deigned to check the logs, I would have seen:

11740 11:10:50 ERROR Cannot create tracker.
Exception: System.Web.HttpException
Message: Unable to serialize the session state. In 'StateServer' and 'SQLServer' mode, ASP.NET will serialize the session state objects, and as a result non-serializable objects or MarshalByRef objects are not permitted. The same restriction applies if similar serialization is done by the custom session state store in 'Custom' mode.

This error will appear in Experience Explorer mode and normal mode, but not in Experience Editor or Preview mode.

Let’s Talk About Session State

When the Experience Database (xDB) was released, there was suddenly a lot of information floating around about session state. In this blog post, I will defeat this nemesis of mine by covering:

  • The role played by session state in the xDB
  • What ‘private’ and ‘shared’ session state means
  • The effect your choice of server architecture has on the way session state must be configured (specifically in-proc versus out-proc)

At time of writing, the most recent release of Sitecore is 8.1 Update-2.

What is session state, again?

If you already know what session state is, skip ahead – or refresh your memory by reading the MSDN article (https://msdn.microsoft.com/en-us/library/ms178581.aspx).

This is how I understand session state: HTTP is a stateless protocol. That means that when I request https://mhwelander.net/ (request #1) and subsequently request https://mhwelander.net/about/ (request #2), the server treats me as a magical and interesting stranger each time. ASP.NET session state is a mechanism by which the server remembers who I am by keeping a short-term record of me. It gives me a session ID when I first start browsing, puts it in a cookie (usually), and hey presto – the server now has a way to identify me as me each time I make a request. Additional data can be added into session state as I make requests – for example, if I use my name to fill in a form, the application might store that as a session state value so that it can add my name into marketing campaigns: “Martina, get 20% off motherboards!”


When we talk about session state in an xDB context, this is what we are talking about – the ASP.NET session state mechanism.

What is ‘out-proc’ and ‘in-proc’?

You can either store session state data in memory (in-proc, or ‘in process’), or you can store it somewhere else – such as in a SQL database (out-proc, or ‘out-of-process’).


In a standard ASP.NET application, there are different out-proc session state providers to choose from – you can see a comparison of the most popular providers here: https://blog.devopsguys.com/2013/07/26/best-performing-asp-net-session-state-providers-2013/.

At time of writing, there are two session state providers available for Sitecore and xDB – one that uses SQL, and one that uses MongoDB. xDB requires that session state providers support the Session_End event, which rules out Redis Cache for the version of Sitecore that is currently available (8.1 Update-2). You should choose the provider that uses a technology that you are able to support and optimize.

What role does session state play in xDB?

Before xDB, there was Sitecore’s Digital Marketing Suite – or DMS (aww, memories). The DMS was a very “chatty” application that made frequent trips to read from and write to the SQL analytics database during the course of a visitor’s session. This had performance implications – particularly for high-traffic, geographically distributed sites where large amounts of data had to travel across huge distances.


By contrast, Sitecore accesses the xDB’s collection database twice during the course of a visitor’s session – once at the start, to identify the visitor and load data into session if the visitor is already known, and once at the end, to flush session data into the collection database. Whilst the session is ongoing, all data – pages visited, goals triggered, patterns matched – are held in session.

Relying on session state to hold important data about visitors and their interactions creates the following requirements:

  • Session state should be managed out of process – this session state resilient against ASP.NET errors and IIS restarts, and it is a hard requirement in a clustered environment with multiple content  delivery servers (I will cover this later)
  • Every content delivery cluster should have a dedicated session state database server
  • The session database server should be as physically close to the CD servers as possible, and on the same network

The following example shows two content delivery clusters, each with a dedicated session state server:


In this scenario:

  • A visitor hits a cluster (e.g. Europe) – a request goes out to the collection database to identify that contact
  • If they are known, data about the contact is loaded into session and a lock is placed on the contact against the current cluster until the end of the session
  • The visitor browses, bouncing between content delivery servers within the cluster (sticky sessions are not required if session state is managed out of process) – data about the visitor’s interaction is written to the session state database
  • After a period of inactivity, the session expires – analytics data is written to the collection database and removed from the session state database, and the lock on the contact against the cluster is released

Private and shared session state

There are two types of session state – private and shared. An easier way to think of this is that there are two types of session state data being stored; data that is private to a particular interaction and data that is shared between interactions that overlap. Private session state contains information about the interaction – such as pages visited and goals triggered on that device. Shared session state contains information about the contact – such as their e-mail address and engagement plan state.

When you install Sitecore locally, private and shared session state are set to ‘in-proc’. Private session state is configured where you would configure regular old ASP.NET session state – in web.config. Shared session state is configured in Sitecore.Analytics.Tracking.config.

This distinction (shared vs private) is required to support a single contact with two concurrent sessions on two different devices – for example, you might start browsing a website on your laptop and continue on your phone whilst the laptop session is still ongoing (maybe you had to pee during the last few moments of an E-Bay bidding war). What happens if you moved to a different engagement plan state during the laptop session, or update the e-mail address that is stored in xDB? The laptop session is still ongoing, so those changes have not made it to the collection database yet.

This is where shared session state comes in, flexing and looking impressive. Contact data needs to be shared across multiple device sessions, and is therefore stored in shared session state. If I moved into a different engagement plan state or changed my name on my laptop, my toilet phone needs to know about it immediately – before that information hits the collection database.

Why is it important to share certain data between sessions? From a purely technical standpoint, you do not want data from session 1 to be lost because session 2, which expires after session 1, loaded out-of-date information from the collection database before it started. The diagram below (from http://doc.sitecore.net/) shows the lifetime of shared session:


From a usability point of view, you are offering a seamless experience across multiple devices – imagine how unimpressed you would be if the personalization rule for a particular engagement plan state was active on your laptop, but your toilet phone acted like it was living in the past.

Read more about private and shared session state on doc.sitecore.net: https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/session_state/session_state

A little note about cluster-forwarding

You may wonder what happens if toilet phone is routed to a different cluster of content delivery servers (let’s say you are in between an east coast and west coast cluster). Worry not – at the start of your laptop session, a lock was placed on your contact data against a particular cluster. When your second session starts, Sitecore checks the xDB and forwards you to the cluster that your pre-existing session is on.

Read more about cluster forwarding on doc.sitecore.net: https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/server_considerations/server_clusters_and_transferring_contact_sessions

When and why do I have to use out-proc session state management?

If you are using the xDB and you have more than one content delivery server per cluster, you MUST USE OUT-PROC SESSION STATE. In-proc is categorically not an option, not even if you use sticky sessions. This is all because of the need to support the sharing of session state information for a contact with two concurrent session on different devices. Let’s look more closely at why you cannot solve the problem with sticky sessions:


I visit a website from my laptop and log in. My login details happen to be my unique identifier in the xDB, and it loads my contact information from the collection database. This website is backed by three content delivery servers – my session sticks to content delivery server #1 and will stay there for the duration of my session. Session state data is managed in-proc; private and shared session state data is written to memory. So far, so good.


I abandon my laptop and switch to my phone – I log in, am identified a second time, and a new session starts. Or does it? If my phone session sticks to a different content delivery server, Sitecore has no way of knowing what is happening to my contact data on content delivery server #1 – because it has no way of accessing the memory (and therefore the shared session data) of another machine. I could have changed my name to Myrtle McMuffin; until that information makes its way into the collection database, my mobile phone has no idea that this has happened.

At time of writing, I do not know what will happen in this scenario (edit: but Tauqir Malik does!)  – either the second session simply cannot get a lock on the contact and the session hangs, or you open yourself up to data inconsistencies when session B overwrites session A, and Myrtle McMuffin disappears forever.


This is why  you must use out-proc session state management in an environment with multiple content delivery servers.

Read more about session state and the xDB on doc.sitecore.net: https://doc.sitecore.net/sitecore_experience_platform/setting_up__maintaining/xdb/session_state/session_state.

Hey – can I mix in-proc for private and out-proc for shared?

Technically yes, since you can configure different providers for shared and private session state, but I cannot point to proof that you gain anything at all from doing so. Shared session state must always be configured to use out-proc session state in a multi-content delivery server environment, and will be your limiting factor.

Update: What happens to session state data if the collection database becomes unavailable?

Please see Jason Wilkerson‘s post on xDB Session Info and Mongo Availability.


Thank you in particular to Dmitry Kostenko and Todd Mitchell for helping me make sense of session state. Diagrams courtesy of SimpleDiagrams.

This is a confusing topic, and I welcome corrections and clarifications! Either post a comment or contact me on Twitter: @mhwelander

Sitecore Azure for Beginners – Part 1: What Is Microsoft Azure?


In this series of blog posts, I will be tackling Microsoft Azure as it relates to Sitecore. At the time of writing, I am a complete newcomer not only to Sitecore Azure but Microsoft Azure in general. This series assumes that you, like me, know nothing at all about Azure beyond the fact that it has something to do with ‘the cloud’.

In part 1, I am going to show you around the Windows Azure dashboard (there is also a preview of the new portal, which will replace the old one soon) and explain the difference between PaaS and IaaS in plain English.


Special thanks to Oleg Burov (Sitecore Product Support Services) and Damian Brooks (Sitecore UK Technical Consulting) for fielding endless questions from me on this subject.

What is Microsoft Azure, and what does it offer?

The introductory blurb about Microsoft Azure is actually quite useful. When you choose to deploy your application to something like Azure, it is because you do not want to be in charge of your own physical kit. That includes hardware, the operating system, network load balancing – things that really have nothing to do with your core application.

There are other cloud platforms out there – like Google Cloud Computing and Amazon Web Services (which focuses on IaaS). They all offer similar services. This series covers Microsoft Azure only.

What does Microsoft Azure look like?

For the purposes of this blog post, I signed up to a trial of Microsoft Azure (it lasts 1 month). They give you a subscription ID. This is what my Management Portal looks like (notice that I have created an SQL Database and a Web Site):


And here is a complete view of the services available:


As a beginner, my first reaction to this list was: where on earth does PaaS and IaaS fit in to this list? I certainly cant ‘create a PaaS. At this point, I am going to take the opportunity to make the distinction between Platform as a Service and Infrastructure as a Service, because it isnt as obvious as I thought it would be.

IaaS and PaaS

IaaS, PaaS, (and indeed SaaS) are choices you make about the level of abstraction you require. They arent things or services in themselves, or something you buy. Your final setup might be (and probably will be) a combination of IaaS and PaaS elements.

Infrastructure as a Service (IaaS) means that you are handing over the responsibility of machinesand network to a cloud platform. Essentially, you are opting for virtual machines over physical machines. You are still able to log in to these virtual machines and do things like configure IIS, mess around in MSSQL, and look at your web root.

Platform as a Service (PaaS) means that you are handing over the responsibility for things like SQL management to the cloud platform by making use of services. At the time of writing, the Sitecore Azure module makes use of five Azure PaaS services: Cloud Service, Storage Service, SQL Databases Service, Traffic Manager Service, and Cache Service to some extent (more on this later – caching is done in the role, but the Cache Service is used to support DistributedCacheSessionStateProvider).

Rather than logging in to a virtual machine to manage these aspects of your application, you are letting the cloud platform do the heavy lifting. As an example, here is a view of the sample database I created. By using this service, ‘scaling is now a matter of making a choice in a dropdown:


Finally, Software as a Service (SaaS) takes things one step further – all you do is sign up. Google Apps for Work is an example of SaaS; you have no idea what goes on behind the scenes, you just care that it works.

There are pros and cons to each approach. The Azure documentation has a very good article comparing websites, cloud services, and virtual machines (all of them are options available through Azure). In each case, the end result is the same – a website, but the level of control you have is inversely proportional to ease of use. This post by nonlinear also does a great job of differentiating between IaaS, PaaS, and SaaS.

Back to Sitecore

If you want to use Microsoft Azure with Sitecore, you have a few options:

  • Use virtual machines. This is the Infrastructure as a Service approach – you will be required to set up a number of content management, content delivery, and SQL servers, just as you would if you owned the hardware yourself.
  • Use Sitecore’s Azure module (link is to version 1.3), which provides a layer of abstraction on top of a number of Azure services. This is the Platform as a Service approach – the module automates the deployment of content delivery and content management farms.
  • A hybrid approach – such as a PaaS approach without using the Sitecore Azure module

Warning: Will PaaS and the Sitecore Azure Module Suit Your Project?

On paper, the Platform as a Service approach using the Sitecore Azure module seems like the obvious choice for all Sitecore implementations – the screenshots literally show an application where you can right-click and ‘Add Editing Farm…’. But please remember that it is not magic!

However, certain implementations arent suited to this level of abstraction. Keep in mind that by opting for a service, you are relinquishing a certain amount of control.

  • Make sure you understand the limitations of the services that the Sitecore Azure module relies on – for example, SQL Azure has limits on transaction log size, which becomes an problem when attempting to rebuild heavily fragmented indexes.
  • Make sure you have a solid understanding of Microsoft Azure in general – especially the services that the Sitecore Azure module makes use of. You can read more about these services at the end of this article. Just as you must understand ASP.NET in order to make use of Sitecore as a developer, it is highly recommended that you understand Microsoft Azure before deploying your solution to the cloud.


In upcoming blog posts, I will be looking in more detail at the Sitecore Azure module, what it does, the services it relies on, and some recommend practices.

Useful Links

Sitecore Azure How Tos

Azure Cloud Service

Azure Storage Service

Azure SQL Databases Service

Azure Traffic Manger Service

Generating meaningful sample Sitecore analytics data with JMeter

Note: Although the screenshots I am using are from the Experience Profile interface, which is not publicly available at time of writing, this method will work with older versions of Sitecore as well.
Edit (05/08/2014): See comments by Alen, Ivan, and Michael regarding robot detection and forcing session end, as these may affect whether or not JMeter activity is excluded from analytics.

The Sitecore Experience Platform combines the CMS with a whole host of marketing and analytics features. In the upcoming 7.5 release, the system architecture that drives these features have been given an overhaul – where previously there was a single SQL database, there is now a more scalable infrastructure that brings MongoDB and search indexes into play.

There is also a new interface – the Experience Profile, which you can read more about here. This interface lets you view contacts and their interactions with your brand in a user-friendly SPEAK interface. You can see the engagement value they have accrued in total and per visit, see which pattern card best matches them based on the pages they have visited, and how they came to the site – to name a few.

I am working with this version at the moment, and it brought up an age-old problem – how do we generate meaningful, fake analytics data? I have previously used the Traffic Generator, but I would like more control than that, and I do not want to generate it all by hand.

Why bother? Well, internally, we need meaningful data for training – you may need it to show customers what they can expect from the interface, or test that your goals and profile points are adding up as expected (I wouldn’t run this on a live site, though, as it will interfere with your analytics data).

Sitecore Support gave me the idea to use JMeter (I know there are plenty of other performance testing tools out there, but this one worked for me). In this blog post, I will show you how to:

  • Generate large amounts of traffic
  • Customize the request header to fake referers, user agents, etc
  • How to fake multiple visits from a single contact – but you still only have to run the script once

A single session with a couple of page visits

First of all, download Apache JMeter. Open it by running the jmeter.bat file in the bin folder – it will open a terminal window and this interface:


Next, we are going to create a thread with a number of page requests.

Create some defaults

Right-click on the test plan and Add > Config Element > Http Request Defaults. Put this at the top of your test plan. This node has many of the same fields as an individual Http Request node, and allows you to set up defaults. The only one I need right now is Server Name or IP, which I will set to http://traincore – now I will need need to specify the host name every time I create a request node. You can override it per Http Request, of course:


Create a thread

Right-click on the test plan and Add > Thread (Users) > Thread Group. Give the thread group a new name – I have called mine Contact Thread 1:


A thread is like a browser session – it is the same thing as you opening your browser, clicking around a bit, and closing your browser again. Change the Number of Threads (users) to 10.

Create a cookie manager

In order for a thread to take cookies into account, you need to create an Http Cookie Manager node. Right-click on the thread thread node OR the test plan node as a whole and Add > Config Element > Http Cookie Manager.


That’s it. The very fact that this node exists in your thread means that the thread will use cookies – which will be very important once we come to generating multiple interactions from a single contact.

Create a number of page request

Right-click on your thread and Add > Sampler > Http Request. This will be your first page visit. In the Path field, type a path to a site page that is relative to the host name you set up in the Http Request Defaults node – e.g. /holidays/battle-of-the-hills/itinerary/day-1.

Give your node a meaningful name (Battle of the Hills – Day 1), and create a few more, each for a different page. I have added three separate requests to simulate the following user simple journey: Holidays > Battle of the Hills > Itinerary : Day 1:


Before we run this test plan, right-click on your thread and Add > Listener > View Results Tree. Ensure that this is at the bottom of your thread. As your thread runs, this node will collect the results:


Now run your test plan by pressing the play button.

JMeter results

Cick on the View Result Tree node. In the Sampler result tab, you can see what came back from the server:


We ran 10 separate sessions, and each session started with a visit to the Holidays page. Click on that, and you should see something like this:

Response headers:
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
Set-Cookie: ASP.NET_SessionId=gsogo1kq3vhymh0togwpt1o0; path=/; HttpOnly
X-AspNet-Version: 4.0.30319
Set-Cookie: SC_ANALYTICS_GLOBAL_COOKIE=10f2535e791043fd9e9e7b67a2e4eb93|False; expires=Tue, 23-Jul-2024 17:26:06 GMT; path=/; HttpOnly
X-Powered-By: ASP.NET
Date: Wed, 23 Jul 2014 17:26:05 GMT
Content-Length: 22440

That is a response header. You can see that the Sitecore analytics cookie is returned to your ‘browser’ (which is JMeter): Set-Cookie: SC_ANALYTICS_GLOBAL_COOKIE=10f2535e791043fd9e9e7b67a2e4eb93|False; expires=Tue, 23-Jul-2024 17:26:06 GMT; path=/; HttpOnly

Click on several Holiday debug entries and confirm that the analytics cookie is new in each case – from Sitecore’s perspective, we have simulated ten different contacts visiting the site.

If you look at the response header for a later step in the test plan, such as Battle of the Hills, you will not see the Sitecore analytics cookie in the respone header, because it has already been set for that session:

HTTP/1.1 200 OK
Cache-Control: no-cache, no-store
Pragma: no-cache
Content-Type: text/html; charset=utf-8
Expires: -1
Server: Microsoft-IIS/7.5
X-AspNet-Version: 4.0.30319
X-Powered-By: ASP.NET
Date: Wed, 23 Jul 2014 17:26:06 GMT
Content-Length: 12383

Viewing this data in Sitecore

Like I said at the start – this method does not require Sitecore 7.5. Let’s look at the Experience Editor interface. The first page lists the most recent visits – we should have 10 separate visits, and they will all look like this:


But they look a bit boring! Each was 0 seconds long, no engagement value was triggered, we know they went to exactly the same pages, and each contact only made a single visit.

Don’t worry – we can fix all of that.

Using a random controller to vary user path

I want my user paths to vary a little bit. Not everyone takes exactly the same path through the content, and I want to simulate that.

However, keep in mind that a visitor’s path is not totally random. Let’s think about the types of people that might visit the holiday site – an expert mountain biker or a dad looking for something family-friendly. I want each session to either simulate a visitor looking for family holidays or an visitor looking for challenging mountain biking holidays.

To do this, I right-click on my thread again and Add > Controllers > Random Controller. A random controller execute one of its child nodes – and any sub-trees it has.

Inside the random controller – create two children of type Simple Controller but right-clicking and selecting Add > Controllers > Simple Controller. Call one Expert and the other Family. Under each of these Simple Controller nodes, build a list of Http Request nodes to simulate a journey for that particular visitor type – a family-oriented visitor might visit ‘Bikes for Families’ and ‘Road Safety’, whilst an expert cyclist might visit ‘Battle of the Hills – Adrenaline Holidays’ or ‘Latest deals on high-performance bikes’:


Remeber, JMeter has no idea what this means, it is only making requestsSitecore is the one that will think this particular ‘visitor’ is from a particular location.

Lastly, to avoid every ‘family’ and ‘expert’ visit being identical, you might want to add a second Random Controller node (before or after the first one) with a number of HttpRequest children that anyone might visit – contact us, special offers, deals. You might even create a number of sub-trees using Simple Controller nodes to simulate a variety of possible paths. This is totally optional:


With enough nested Random Controller nodes, you can start to create journeys that look nothing alike. Run your test plan again and look at the View Result Tree node (it is best if you change the thread user count back to 1 and run the plan several times) – each journey is slightly different. Your analytics will reflect this.

Adding pauses

But every visit is still zere seconds. You can change that by adding some random pauses to your test plan in the form of Test Action nodes (Add > Sampler > Test Action). This node allows you to specify a Pause in milliseconds. Sprinkle randomly throughout test plan, and use Random Controller nodes to ensure that every user’s visit time is slightly different.


Triggering some goals

What about engagement value? Well, we know that a contact’s engagement value depends on which goals they trigger.

You could trigger goals the way a human being triggers them – by downloading files, filling in forms.. or you could just trigger them with sc_trk={GUID} (this query string may be different depending on what your configuration is set up to do).

To do this, your HTTP Request node needs to have a parameter specified (the query string key) and a value (in our case, the GUID for the goal. Because the GUID includes curly braces, make sure you tick ‘encode’:


You can see what your request looked like by using the View Result Tree node, and looking the Request tab for the request that had the query string.

Again, I am going to put a number of goal triggering requests under a Random Controller to give me some variety.

My visits now have varying durations, engagement value, and pages visited. But I still only have a single visit per visitor.

Faking multiple visits from the same contact

How does Sitecore know you are a return visitor? It uses the SC_ANALYTICS_GLOBAL_COOKIE cookie. This cookie is set when you first visit a Sitecore site, with an expiry date set far in the future – ten years, or something like that. When you close your browser, the browser session ends – but your cookie is still there. When you visit again, Sitecore recognizes that you have visited the site before.

This does not work very well for developers, who clear their cache several times a day or refuse to accept cookies – but a normal internet user doesn’t actually bother clearing their cookies all that often (true story).

How do we use the same cookie over multiple threads in JMeter?

There is a way to share variables across threads in JMeter. I took advantage of this. First of all, I duplicated my entire thread a few times – and changed aspects of it (different ‘Family’ and ‘Expert’ paths, different goals, more Random Controller nodes). I even created a Http Request Header node in each thread, and specified a different Referer for each. This will vary the ‘traffic source’ property for each visit.

Let’s say we have four threads. In each of the threads following the first one, I want there to be a 50/50 chance that the cookie from the first visit will be re-used.

To do that, I first need to extract the analytics cookie from the first page request in the first visit.

Extracting the cookie

Enter PreProcessor and PostProcessors. They allow me to do things before and after a request has been made. My first Http Request node in the first thread is to the Holidays page. I am going to Add > PostProcessor > Regular Expression Extractor to that:


This extractor will allow me to look at the response (header or body) and extract what I need using RegEx. I want everything that follows SC_ANALYTICS_GLOBAL_COOKIE in the header. I am not particularly good at RegEx, but this worked for me:


Here is the whole thing:


Once extracted, this value is stored as the variable name you specify in the Reference Name field – I chose AnalyticsCookie.

Making the cookie available to other threads

Now that I have my cookie, I want to make it available to subsequent requests in my test plan. JMeter has something called global variables, which allow me to set variables in one thread and retrieve them in the other.

To do this, I am going to create another PostProcessor node that runs immediately after the Regular Expression Extractor – a BeanShell PostProcessor. BeanShell is dynamically interpreted Java (with some extra ‘stuff’).

This postprocessor is going to retrieve the locally scoped variable and set it to a global variable. This is the script:

import org.apache.jmeter.util.JMeterUtils;

var analyticsCookieLocal = vars.get("analyticscookie");

JMeterUtils.setProperty("analyticsCookie", analyticsCookieLocal);

print("Cookie set: " + JMeterUtils.getProperty("analyticsCookie"));

And as a screenshot:


I have used print to show what is going on in the terminal window that opened up when I started JMeter. As your test plan runs, it will print out ‘Cookie set: (COOKIE)’ each time the script runs.

Now that we have set the cookie, we need to move on to our subsequent threads and randomly choose whether the cookie is re-used or reset.

Re-using or resetting the cookie


Go to the second thread in your list. Make sure this thread has an Http Cookie Manager node.

The first thing we want to do in this new thread (which represents a person opening a new browser window) is to either send the existing analytics cookie with the first request or accept a new cookie from the server.

To do this randomly, create a Random Controller node. Inside it, create at least two Http Request child nodes – they do not have to be for the same page.

As we know, the random controller will choose one of its children to run. We also know that there is such a thing as pre-processors, that run before a request is executed. Assuming you have two Http Request nodes, add a BeanShell PreProcessor node as a child to each one.

Here is the result:


In one of the pre-processors, simply print a message:

print("NO RE-USE");[/java]

If the random controller chooses that Http Request node, all the pre-processor will do is print out a message - it will then hit the first page and a fresh analytics cookie will be returned.

In the second pre-processor script, <strong>extract and set the cookie stored in the global scope</strong>:

import org.apache.jmeter.util.JMeterUtils;
import org.apache.jmeter.protocol.http.control.CookieManager;
import org.apache.jmeter.protocol.http.control.Cookie;

// Get context cookie manager
CookieManager manager = cm=ctx.getCurrentSampler().getCookieManager();

// Create new cookie, retrieve global variable and set as value
Cookie cookie = new Cookie("SC_ANALYTICS_GLOBAL_COOKIE",JMeterUtils.getProperty("analyticsCookie"),"traincore","/",false,0);

print("RE-USING THIS COOKIE: " + JMeterUtils.getProperty("analytics"));



This script retrieves the current Http Cookie Manager in code and creates a new analytics cookie from the global variable you set in the first thread. This script will run before the request does, setting the cookie, which is then passed to the server (you can see this happening if you use a View Result Tree). Sitecore now thinks you are returning with the same analytics cookie, and will count you as an existing, anonymous contact, and a second visit.

If you duplicate the second thread a few times - the one with the cookie-setting pre-processor - there is a 50/50 chance in each thread that the cookie will be re-used. You will see a contact visit count with between 1 and n, where n is the number of threads. Provided you only have two options for your starting page in each thread, there is a 50% chance that a contact will make 2 visits, a 25% chance they will make 3 visits, a 12.5% chance they will make 4 visits, and so on until infinity (well, n).

Sitecore now thinks I'm a repeat visitor:


You can use this method with other cookies (ASP.NET session, for example), although I believe there is a smarter way to authenticate users.

We now have contacts making multiple visits, of varying value and duration, following different paths. Already, the data is starting to look more realistic. What else can you do?

More options for faking analytics data

The Http Request Header tells you a lot about a visitor. JMeter has a configuration node called Http Request Header that allows you to fake things like the Referer (which site did your user browse to your page from?) or X-Forwarded-For(which IP did this person come from?). You can set the request header defaults per thread or for an entire test plan.

You could also pass other global varibles between threads. Currently, I am getting JMeter to store the 'visitor type' path a visitor took, saving that as a value ('family', for example), and using it to determine what type of content the visitor hits in subsequent threads - if the cookie is re-used, because that means it's the same visitor. The result is that someone who visited 'family' holiday pages in their first visit will not suddenly be visiting pages about expert bike repair.


Splendid cookie by http://thenounproject.com/term/cookie/37916/

SPEAK for Newbies – Part 6: SPEAK Helpers

The main SPEAK .js file contains a lot of very useful helpers that you can use in your own applications and components. To remind you, the PageCode component uses require.js to load in a bunch of scripts. The ‘Sitecore’ .js file is highlighted below.

(function (global) {
    baseUrl: "/sitecore/shell/client/Speak/Assets",
    paths: {
      jquery: "lib/core/deps/jQuery/jquery-1.10.2",
      underscore: "lib/core/deps/underscore/underscore.1.4.4",
      knockout: "lib/core/deps/ko/knockout-2.2.1",
      backbone: "lib/core/deps/backbone/backbone.1.0.0",
      sitecore: "lib/core/sitecore-1.0.2",
      sitecorify: "css/sitecorify",
      bootstrap: "lib/ui/bootstrap",    
      jqueryui: "lib/ui/deps/jQueryUI/jquery-ui-1.10.1.custom",
      dynatree: "lib/ui/deps/DynaTree/jquery.dynatree-1.2.4",
      dynatreecss: "lib/ui/deps/DynaTree/skin-vista/ui.dynatree"
    shim: {
      'jquery': { exports: 'jQuery' },
      'jqueryui': { deps: ['jquery'] },
      'underscore': { exports: '_' },
      'knockout': { deps: ['underscore'], exports: 'ko' },
      'backbone': { deps: ['jquery', 'underscore'], exports: 'Backbone' },
      'sitecore': { deps: ['backbone', 'knockout'], exports: 'Sitecore' },
      'dynatree': { deps: ['jqueryui'/*, 'css!dynatreecss'*/] }
    map: {
      '*': {
        'css': 'lib/core/deps/css'

  require(["sitecore"], function (_sc) {

The ‘sitecore’ module is used by all page code (and presumably all components). To remind you, here is the first line of my page code from part 5:

define(["sitecore", "jquery"], function (Sitecore, jQuery) {

As you can see, the ‘sitecore’ module is injected into the function as ‘Sitecore’ (you could just as easily have called it _sc, sc, scspeak). It’s this magical object that will give you access to a bunch of really useful helpers. Let’s have a look at what’s available (from somewhere in the depths of sitecore-1.0.2.js):

_.extend(_sc, {
  Helpers: {
    url: urlHelper,
    date: dateHelper,
    id: idHelper,
    string: stringHelper,
    object: objectHelper,
    invocation: invocationHelper,
    overlay: overlayHelper,
    window: windowHelper


You can use the helpers in your own application. Below is a list of some of the helpers (I isolated the ones I felt were most useful):

Helper Usage
combine Combine two ‘URL’ strings – will put a leading / in front of everything, so shouldn’t be used with domains.

var combined = Sitecore.Helpers.url.combine('/my/path/to/something/', 'else');
var url = "http://www.mhwelander.net/?isSwedish=true;
var isInUrl = Sitecore.Helpers.url.isParameterNameAlreadyInUrl(url, 'isSwedish');
var originalUrl = "https://mhwelander.net/";
var processedUrl = Sitecore.Helpers.url.addQueryParameters(originalUrl, {
    isSwedish: true
var parameter = Sitecore.Helpers.url.getQueryParameters(window.location.href)['parametername'];
Helper Usage
isId The following will return true:

var isId = Sitecore.Helpers.id.isId('{EDF76D4D-E94B-4BAA-8CC7-AADCBB33092E}');
isId This accepts a string formatted as a short ID, which is basically a GUID without the ‘{‘, ‘}’, and ‘-‘ characters:

var isId = Sitecore.Helpers.id.toId('EDF76D4DE94B4BAA8CC7AADCBB33092E');
toShortId This accepts a string formatted as a GUID:

var isId = Sitecore.Helpers.id.toShortId('{EDF76D4D-E94B-4BAA-8CC7-AADCBB33092E}');
Helper Usage
endsWith The following returns true

var endsWithWelander = Sitecore.Helpers.string.endsWith('Martina Welander', 'Welander');
equals The following returns true(not case sensitive):

var equalsMartinaWelander = Sitecore.Helpers.string.equals('Martina Welander', 'Martina Welander');
format Works a bit like String.Format – use sparingly.

var formattedString = Sitecore.Helpers.string.format('{0}, {1}', 'Welander', 'Martina');
Helper Usage
getOwnProperties The following returns an array of properties:

var person = { name: 'Martina Welander', nationality: 'Swedish', occupation: 'SPEAK minion' };
var arrayOfProperties = Sitecore.Helpers.object.getOwnProperties(person);

There are others – you can test them all out by using your browser’s developer tools. The main application is available as _sc:


Sitecore’s marketing features for developers: What is ‘Engagement Value’?

Martina Does Marketing

If you work as a developer in the Sitecore world, you will probably hear the term engagement value at some point. I did a quick Google (and got a 500 error, which made me think the world was coming to an end, but I’ve recovered from that now) and noticed that the majority of resources out there are geared towards marketers – and fair enough; it’s a tool for marketers.

But as a developer, I think it can be easy to forget that an enormous part of the Sitecore platform is dedicated to optimizing the experience for visitors – whilst I’m silently cursing the existence of the TemplateID class and crying over line 230 of some repository layer, someone else is trying to figure out how best to gauge a visitor’s wine preferences and personalize their experience accordingly.

Engagement value sits at the core of that. In this post, I’m going to explain what engagment value is,who is responsible for it, and how it impacts a developer’s job.

Engagement Value vs Traffic

Imagine that you run a site that sells cheese and wine. You think you’re popular – you get 10,000 visitors per day. However, 1 in 10,000 visitors actually buys anything. Your traffic is high (maybe you are some sort of SEO magician), but based on the sales metric alone, their engagement with your site is very poor. Traffic isn’t an indicator of how much money you are making – you need information about what those visitors are actually doing on your site. This is where engagement value comes in.

Engagement value, goals, and the engagement value scale

In order to determine what your visitors are doing and how much that is worth to your business, you need to set up some goals. A goal corresponds to an action taken by the visitor that suggests they are engaging with your site. In our wine-and-cheese example, this might include:

  • Registering for the newsletter
  • Downloading a special offers coupon
  • Rating a product
  • Signing up for the ‘Guess that Cheese’ competition
  • Purchasing a product

Some of those goals are more important than others. If a visitor downloads a special offers coupon, they are clearly interested in purchasing whatever that offer entails them to – but it isn’t as important as actually parting with cold, hard cash. The more valuable a goal is to your business, the higher the engagement value associated with that goal will be.

A goal is actually an item in Sitecore, and as a developer, you can either assign it as an ‘event’ on a page, or trigger it programmatically. Here are some goal items:


Every goal has an Points field. This is the engagement value of the goal:


As a visitor navigates around the site, they trigger goals and accrue value for that visit. Marketers can use the Executive Insight Dashboard to see reports on traffic vs engagement:


The bars represent total engagement value. If this was the data for our wine and cheese shop, we can see that people are engaging with our brand – in fact, email is generating rather a lot of engagement! Our sales might be low, but our marketing efforts have not been not a complete disaster – people are signing up for the newsletter, downloading special offer vouchers, and rating our products. Marketers can use this data to fine-tune their approach – why is nobody coming in from Facebook or Twitter? Have we omitted these channels? Is it really worth paying for advertising? Suddenly, you have a much more nuanced view of what is happening across the various channels available to you as a marketer.

That’s all well and good – but what is my job in all of this, as a developer?

Who decides how much a goal is worth?

Deciding the value of a goal is definitely not a developer’s job. This is something that the business must decide, and it is recommended that they attend a scoring workshop (run by Sitecore Business Optimization Services – or ‘SBOS’) to work that out. As a developer or system administrator, you should expect to be given a list of goals and values to set up in Sitecore, and instructions on when and how these goals should be triggered.

Configuring goals

OK – I’ve been given my list. How do I actually configure goals?

Goals are stored in the Marketing Center as Sitecore items. You can get to it via the Sitecore Desktop by clicking Sitecore > Marketing Center:


This will open the ‘Marketing Center’ portion of the Sitecore tree in a separate window. Alternatively, you can locate the Marketing Center via the Content Editor – it’s under /sitecore/system/Marketing Center.

Right-click on the Goals item and insert a new goal:


Note: At time of writing, I believe that putting your goals into subfolders (goal categories) might make it impossible to select them in Web Forms for Marketers (which does not expect sub-folders), so you may find that your goals need to be on one level.

Give your goal a name and fill in the Points field with the engagement value given to you by the business. The item will enter into a workflow. When you are happy with your goal, you must deploy it – basically push it into the ‘final’ state of the goal workflow:


Behind the scenes, deploying a goal creates an entry in the analytics database for that goal. Tip: If you are using continuous delivery to deploy your site each time you make a code change, you will need to ‘re-deploy’ existing goals to make sure this row is created – see Andrew Thompson’s blog post for more details on goals and continuous delivery.

And that’s it – your goal is now live and ready to be used.

Coming up with a sensible engagement value scale and setting it up in Sitecore is one of the first things marketers will do as they start getting to grips with the Sitecore platform; even if it takes a little while to get going with personalization, testing, and content tagging, you are gathering data in that time.

Triggering goals

Goals can be triggered in a number of ways. In reality, although there is a way to assign goals to page items in the Content Editor, most goals are triggered programmatically. A visitor might land on your ‘contact us’ page, but they have not really engaged unless they clicked the email link or submitted a form.

More often than not, you will find that the developer is called upon to set up goal triggers using the API. Here is how (via @anthonyhook:

if (Sitecore.Analytics.Tracker.IsActive && Sitecore.Analytics.Tracker.CurrentPage != null)
	var pageEventData = new PageEventData("GoalName")
		ItemId = Sitecore.Context.Item.ID.ToGuid(),
		Data = Sitecore.Context.Item.Paths.ContentPath,
		Text = Sitecore.Context.Item.Name


[b]Note:[/b] Since the goal name is stored as a string, you need to make sure that you do not have duplicate goal names.

Finally, in case you were interested, goals can be assigned to entire pages by going to the Analyze tab and clicking the Goals button:


You can assign any number of goals to an item. I will reiterate that whilst this is a simple, UI-based way to assign goals to items, simply viewing a page is rarely a sign that someone is engaging.

Web Forms for Marketers

Web Forms for Marketers (a Sitecore module that allows editors to build forms without help from programmers) is set up to integrate with Sitecore’s marketing and analytics features, and will give you the option to choose a goal to trigger when a visitor completes a form:



Traffic does not give you very much information about how a person interacted with your business, which is why Sitecore really pushes engagement value as a way for marketers to understand what their visitors are actually doing. Because this engagement value forms the core of how success is measured, most other marketing features will be based around engagement value – for example, the success of a test variation is measured in how much engagement it resulted in, not the number of clicks.

As a Sitecore developer, you will very likely be asked to trigger goals via the API at some point, and you may even end up wrapping the goal-triggering mechanism in a web API of some kind.However, the actual goals and the associated engagement value scale is something that the business stakeholders need to work on together with their Sitecore partner’s optimization experts. You can help your client by being aware of the options for triggering a goal, and the fact that certain modules will do this work for you automatically – like WFFM.