Who does IBM Watson say that I am?

Recently I came across IBM Watson Personality Insights. This purports the following:

Personality Insights extracts personality characteristics based on how a person writes. You can use the service to match individuals to other individuals, opportunities, and products, or tailor their experience with personalized messaging and recommendations. Characteristics include the Big 5 Personality Traits, Values, and Needs. At least 1200 words of input text are recommended when using this service.

I thought it would be an interesting exercise to feed IBM Watson various blocks of my texts to see how consistent its findings are, across a range of texts. The demo is available here

First I tried 10706 of fiction. This is from an ongoing work in progress, or rather a piece I have worked on and abandoned and restarted several times. It is not a coherent narrative but lengthy passages of coherent narrative strung together.

Anyhow, here is what Watson said, and given the word count it deemed the analysis “very strong”:

Summary

Sounds plausible, although I am not sure I quite so unconcerned with “both achieving success and tradition.” A few specific likes and dislikes are featured:

You are likely to______
– like country music
-be concerned about the environment
-read often
You are unlikely to______
-like rap music
-like romance movies
-eat out frequently

At various stages of life I have eaten out frequently…. but not anymore, more through the circumstance of having three young children than anything else. Otherwise, fairly plausible. And there are also percentiles on various traits given. I think this post could degenerate even further into navel gazing if I include those, so I will move swiftly on to another specific text, this time my post on Personas for Electronic Health Records on the CCIO website. This is only 807 words, so is counted as “weak”:

You are shrewd.

You are authority-challenging: you prefer to challenge authority and traditional values to help bring about positive changes. You are assertive: you tend to speak up and take charge of situations, and you are comfortable leading groups. And you are self-assured: you feel you have the ability to succeed in the tasks you set out to do.

Your choices are driven by a desire for discovery.

You are relatively unconcerned with both tradition and achieving success. You care more about making your own path than following what others have done. And you make decisions with little regard for how they show off your talents.

Again, plausible enough, although I am beginning to find this uncannily like cold reading. The likes and dislikes;

You are likely to______

be concerned about the environment

read often

be sensitive to ownership cost when buying automobiles

You are unlikely to______

Hmmm – I do like country music. In moderation. The “likely to”s seem OK, the “unlikely to”s less so.

Anyhow, time I suppose to feed another long-ish piece to Watson. So here is my post on The Lonely Planet Guide to Experimental Travel. from the SAU Blog Sept 2005. At 2452 words, this is a “decent analysis”:

You are shrewd and skeptical.

You are philosophical: you are open to and intrigued by new ideas and love to explore them. You are authority-challenging: you prefer to challenge authority and traditional values to help bring about positive changes. And you are solemn: you are generally serious and do not joke much.

Your choices are driven by a desire for discovery.

You are relatively unconcerned with both tradition and taking pleasure in life. You care more about making your own path than following what others have done. And you prefer activities with a purpose greater than just personal enjoyment.

The “likely to” is much less plausible this time:like musical movies (yes), have experience playing music (tragically not), read autobiographical books (sometimes, but not that habitually) The “unlikely to” is a mixed bag:  like country music (see above)
be influenced by social media during product purchases (I would like to think so)
be influenced by brand name when making product purchases (I would like to think so… but that may be wishful thinking)

This could go on all night… but I will wrap up by looking at a post from A Medical Education, the parallel blog to this which features more (somewhat) medically related writing. I have often reflected on whether these two blogs reflect a wider duality or tension within me. I am sure they do. Anyhow, this piece on a particular project I was involved in met the wordcount (although quite a bit of the post is quotation)

At 2622 words, another “Decent Analysis” And what do you know, it is pretty much identical to the prior example based on the Lonely Planet book :

You are shrewd and skeptical.

You are authority-challenging: you prefer to challenge authority and traditional values to help bring about positive changes. You are philosophical: you are open to and intrigued by new ideas and love to explore them. And you are energetic: you enjoy a fast-paced, busy schedule with many activities.

Your choices are driven by a desire for discovery.

You are relatively unconcerned with both tradition and taking pleasure in life. You care more about making your own path than following what others have done. And you prefer activities with a purpose greater than just personal enjoyment.

The only difference is “energetic” replaces “solemn”, and the order of the some of the points. And the likelys and unlikelys are identical to the prior piece.

I haven’t got into the various percentiles for the sake of space, but this is where the surprises have really been. I am coming up in the highest percentiles for “Openness” consistenly, and much lower ones for “Agreeableness.” I also think I value tradition much more than what this would indicate…. but perhaps tradition is not what it used to be, and the traditions I am interested in conserving are no longer continuous traditions in any case.

 Is the whole thing accurate? IBM are at pains to point users to the “science behind the service” 

A well-accepted theory of psychology, marketing, and other fields is that human language reflects personality, thinking style, social connections, and emotional states. The frequency with which we use certain categories of words can provide clues to these characteristics. Several researchers found that variations in word usage in writings such as blogs, essays, and tweets can predict aspects of personality (Fast & Funder, 2008; Gill et al., 2009; Golbeck et al., 2011; Hirsh & Peterson, 2009; and Yarkoni, 2010).

IBM conducted a set of studies to understand whether personality characteristics inferred from social media data can predict people’s behavior and preferences. IBM found that people with specific personality characteristics responded and re-tweeted in higher numbers in information-collection and -spreading tasks. For example, people who score high on excitement-seeking are more likely to respond, while those who score high on cautiousness are less likely to do so (Mahmud et al., 2013). Similarly, people who score high on modesty, openness, and friendliness are more likely to spread information (Lee et al., 2014).

 Amongst other things:

To compute the percentile scores, IBM collected a very large data set of Twitter users (one million users for English, 100,000 users for each of Arabic and Japanese, and 80,000 users for Spanish) and computed their personality portraits. IBM then compared the raw scores of each computed profile to the distribution of profiles from those data sets to determine the percentiles.

Of course, this is using a selected group  – i.e. Twitter users – whose personality portraits may not be representative. There is also a marketing bias to some of the language used which I can understand from a commercial point of view. I would be curious to submit different texts from different times in my life.

Speaking personally, there is a consistency to the verbal reports, but also differences. One does have to wonder how much depends on the type of prose being submitted, and there does seem to be a fiction/non-fiction difference. I don’t seem to be that big on personal enjoyment… and as mentioned before, the lower percentiles for things like tradition, personal enjoyment and other factors are surprising to me. There is a rather unpleasant fixed-in-a-formulated-phrase quality to the whole experience.

Finally (I promise) one can analyse one’s Twitter personality.  This time IBM  had 14394 words to analyse – “very strong analysis” And here is the result:

You are inner-directed and tranquil.

You are authority-challenging: you prefer to challenge authority and traditional values to help bring about positive changes. You are independent: you have a strong desire to have time to yourself. And you are philosophical: you are open to and intrigued by new ideas and love to explore them.

Your choices are driven by a desire for discovery.

You are relatively unconcerned with both taking pleasure in life and tradition. You prefer activities with a purpose greater than just personal enjoyment. And you care more about making your own path than following what others have done.