Comments on: Machine-Based Sentiment Analysis is Flawed

By: marlinex (Marilyn Clark)

marlinex (Marilyn Clark) — Tue, 02 Jun 2009 03:55:06 +0000

Twitter Comment Machine-Based Sentiment Analysis is Flawed [link to post] [Not convinced yet that humans can be replaced in this case] - Posted using Chat Catcher

By: larry levy

larry levy — Thu, 21 May 2009 18:47:03 +0000

Mike – Most people well versed in the space recognize this as a tough problem to solve but still worth trying.

We’ve actually found that the results based on traditional vs non traditional don’t vary as much as you’d expect. This space is being somewhat tainted with companies/programmers just slapping a dictionary onto entity extraction and thinking they’ve nailed the problem.

As we improve our algorithms, we’re achieving results in the high 80%’s in terms of accuracy on negative and positive. It’s the neutrals that cause the most grief – even human inter annotator agreement on these is very low – somewhere between 55%-60%.

By: paullyoung (Paull Young)

paullyoung (Paull Young) — Wed, 20 May 2009 17:23:06 +0000

Twitter Comment I agree wholeheartedly with @mmanuel "machine based sentiment analysis is flawed" [link to post]

By: Rich Reader (WOM-buzz)

Rich Reader (WOM-buzz) — Wed, 20 May 2009 17:00:25 +0000

If you have a large and persistent data stream, do you then obtain stable classification error rates (type A and B)? If so, how high a pair of error rates leaves you better off than human-based coring and classification? If not, how do you place a value on the output?

By: mike manuel

mike manuel — Wed, 20 May 2009 16:56:33 +0000

yeah, i wasn’t implying it’s an all or nothing thing, kevin, just that if you’re investing solely in machine based analytic systems and services, just understand they’re over-promising and under-delivering right now.

By: mike manuel

mike manuel — Wed, 20 May 2009 16:52:04 +0000

agreed, i think the ‘hybrid’ approach (to tac’s point above) is our best bet, but for you, me and so many others to know this and still see so many companies hucking sentiment analytics as a ‘solved problem’ is both the disturbing and flawed part of it all…

By: Kevin Murphy

Kevin Murphy — Wed, 20 May 2009 16:49:32 +0000

Some analyis and listening is better than none. Automated sentiment scoring is a great flagging tool for spotting key issues. But as a measurment tool, it sstill needs to eveolve.

By: Mike Driehorst

Mike Driehorst — Wed, 20 May 2009 16:46:50 +0000

Of course it’s flawed — both the attempt and the way companies promote that service.

However, having a machine do it is a good starting point for us people to then dig into the results to determine their validity (how close or if the machines are accurate).

To do it in large scale by people is very time-consuming when having the machines take the first step followed by people power is a step in the right direction.

By: Rich Reader (WOM-buzz)

Rich Reader (WOM-buzz) — Wed, 20 May 2009 16:33:14 +0000

for the moment, people are still necessary (best qualified) for the scoring and classification components of sentiment analysis. If this moment should pass, then judgment day will have arrived (Terminator-style), wherein sentiment analysis will cease to be a service that is sought-after, and there will be no bail-outs.

By: Tac Anderson

Tac Anderson — Wed, 20 May 2009 16:04:03 +0000

It’s the classic tech biz problem of trying to provide customization at scale. The analytics will get better to incorporate these shifts in Web comm but they will always be behind and will always require a certain amount of human filtering/qualifying.