This is a guest blog post from Dominique Lahaix, CEO of eCairn. eCairn Inc. (@eCairn) is a privately held software technology company, founded in October 2006 that specializes in community and influencers marketing. Headquartered in Los Gatos, California, Dominique (@Dominiq) and other company founders have a cumulative experience of 50+ years in e-marketing, knowledge engineering, collaborative filtering, linguistics, software development and software engineering at well-known companies such as Hewlett-Packard, Sun, Xerox, eBay and Motorola.
Let’s start with an example.
Imagine you work for Dell and you have to rate the following conversations to be examples of positive, neutral or negative sentiment:
- HP is good.
- Dell is good, but I still prefer HP.
- The new Dell PC is good but the previous one worked better.
- The new Dell PC is good but I miss the look of the previous one.
- HP works great but I hate them.
- Dell works great but I hate them.
- Dell is as good as HP.
- HP and Dell are the same crap, maybe Dell a little bit less.
- HP and Dell are nice entry-level products.
- Dell is good if you can afford it.
- Dell is good but exclusive.
- I would only recommend Dell’s PC to small businesses.
- Dell is the Dom Perignon of netbooks.
- Apple is to Dell what Saint Amour is to Beaujolais Nouveau.
- I worked too much on my Dell last night and I got sick looking at the screen.
- Dell is only good for gaming.
- HP is like it was in the Packard’s time.
- HP is like what it was in Carly’s time.
- No wonder why the Dell stock is going south.
- I love HP (from HP’s PR agency or Director).
- I hate HP (from an employee recently fired).
- A tweet repurposing the one above without any additional comment.
Hard to rate isn’t it? These are fairly standard sentences, not corner cases. There is no irony and no borderline use of language, but it just shows that:
- Sentiment is subjective.
- Sentiment is role based: depending on whether you’re the brand manager for Dell overall, or in charge of the new Dell notebook, or the Small Biz vertical, QA manager, Investor relations, or looking into competitive analysis, you may have different views on what positive means.
- Language to express sentiment is context dependant and cultural (wine analogy).
- Language to interpret sentiment is context dependant and cultural (see the HP history analogy).
- Sentiment analysis has to factor in the “who,” and account for real and pseudo duplicates. Same conversation from different persons, similar conversations but from the same person. Everybody is repurposing nowadays, re tweeting, using ping.fm or alike, not counting the huge amount of “robot “sites or blogs that just sit there to attract traffic and ads.
- “Social Media” is not a valid sample of any user base, even when the user based is defined as internet users. As an example: the open-source/Linux community is way more vocal than the windows one. Just taking data from the river of news from community a statistical heresy, except if you want to study Linux fans.
That’s why solutions claiming to do automatic sentiment ends up with 60+ neutral, 70% precision and manual options to override the machine produced sentiment.
It just does not make sense. Putting everything neutral may well be a better bet from a recall and precision standpoint.
I’m not saying it can’t be done. It just can’t be done “generically.” Now if you build a solution for sentiment analysis specializing in the stock exchange community, this is another story. You can build up dictionaries, invest in a learning algorithm, train it… and yes, that sounds doable but it would be very expensive to set up and the applicability of this would be limited to “stock exchange,” even maybe to stock exchange in 2010.
So forget this Holy Grail, stop wasting $$$ on low quality results and come back to the initial objectives of the brand:
Get a sentiment on its brand from a specific audience with a reasonable investment.
There are alternatives to reach this goal and the good news is that math comes to the rescue!
- Why not sampling? Marketers have always used focus group and samples, why not extending that on the social web?
- Why not rating manually? When you zoom in a specific community, you can start doing that. We looked at the top Mommy bloggers that we’ve mapped (top 3,500) and went down to 2000 discussions about Pampers in the last six months. One can do a good job rating three conversations per minute, so it’s roughly a 12 hour job, at $20 per hour, that’s $240 over six months.
- Moreover, as there is more consistency between the conversations that you rate, the quality of the rating would be higher. It’s easier as an example to establish a standard to rate blog conversations from Mommies on Pampers, than to build one for any conversation on Pampers (that would go for analyst reports on Procter & Gamble to sustainability conversations from experts!). Conversations are more consistent, more alike and easier to rate with a focus. You also get specific results that are specific to your key target communities and way more actionable results.
- While you rate, you also spot insights, key conversations to share, ideas for content marketing and more.
- Also, doing it this way, you will actually find your promoters and detractors and build specific outreach plans.
- And as you connect conversations to people, you will see who’s moving in positive territory and can link that to your specific action plans, see whether cluster of influencers are moving in the right direction.
In addition, I’m still wondering what type of actionable plan a brand can take when its “sentiment” drops by 3% when accuracy is claimed to be 70% maximum… The Motrin case went wild over the weekend, Domino Pizza within hours. So for crisis management, investing in Proactive ORM and building up a solid base of fans within the target community is a much better option (Ford’s approach).
And last but not least, when 93% of consumers say they want brands to engage in Social Media, I doubt that they mean engaging with algorithms and I bet they are expecting real and empowered persons. But that’s another story.
