IBM Watson's Tone Analyzer

A few weeks ago this article about IBM Watson's new tone analyzer made the rounds. The goal of the tone analyzer is to help writers "assess and refine" the tone of their written communications. The analyzer considers three different "types" of tone: emotional tone (e.g. anger, cheerfulness), social tone (e.g. openness, agreeableness) and writing style tone (e.g. analyticalness, confidence). Per IBM, the analyzer is currently in "experimental" mode. 

From what I can tell, the analyzer appears to compile scores based on the particular words used and not other factors, such as sentence structure and sentence and paragraph length, that I think are also important in this type of analysis. 

I was skeptical, but thought I'd give the analyzer a go using different types of documents. First, I used the text of an email I sent to a friend. As expected, my emotional cheerfulness score was high (82%), as were my social agreeableness score (84%) and writing analytical score (82%). After all, I was talking to a friend! 

I was surprised to see, though, that my writing tentativeness score was also high (90%), given that I generally make quick, definitive decisions. But when I went back and looked closely at the content of my email, I realized that I did sound somewhat tentative about several topics, though I didn't intend to come across that way.  

Next, I plugged in the text of a professional letter I had sent. My emotional tone showed a high cheerfulness score (78%) but also a really high negative score (92%). I did not intend to sound negative, and in re-reading my letter, I do not believe I sounded negative. The words the analyzer flagged as indicating negativity did not (at least to me) convey negativity given the manner in which I used them, though they could have conveyed negativity in other contexts. My social tone showed scores in agreeableness (55%) and conscientiousness (16%), which I could see in the language I used. And I was pleased to see that my writing style confidence score was 100% as I wanted to (and apparently had) come across as completely confident in my position.  

Finally, I entered several paragraphs of a brief I recently wrote. I anticipated this sample would show confidence, analyticalness, and, perhaps, some unintended hostility. 

The results weren't quite what I expected. The analyzer showed a somewhat negative emotional tone (46%) but also cheerfulness, which I don't really see in the input sample. My emotional anger score was 0%, which I was pleased to see as I didn't intend to come across as angry in my advocacy efforts and was glad I didn't.

My social tone showed 59% conscientiousness, which is good for a lawyer, given that the concept encompasses organization and thoughtfulness. Finally, my writing style was 86% analytical (which you'd expect from a legal document), but also 35% confident and 33% tentative.  I would have expected my confidence score to have been substantially higher and my tentativeness score to have been substantially lower. And in looking back at my sample, I don't see tentativeness or hedging and see language showing substantial confidence in my position.

In my three samples, I agreed with Watson's analyzer in many instances but disagreed in others. This could mean several things: Either the analyzer is imperfect (probable), I'm imperfect at assessing my own writing (also probable) or some combination of the two caused the results (the most probable, I think). 

The analyzer is still in its testing phase, and I expect IBM will continue to improve it. That said, the computer-generated results I got were, in many cases, consistent with my human assessment. And I didn't test the analyzer using someone else's work, only my own. So the disconnect could be the result of my bias toward my own writing, and my results and Watson's results might be more consistent if I hadn't analyzed my own work. 

The analyzer has many potential applications in the world of legal writing, and I'll continue to follow IBM's product updates. I'll be interested to test the Watson analyzer again once the final product is available.