Automated sentiment analysis is becoming more popular for media measurement as the computer-based software programs improve and organizations seek to analyze enormous amounts of mentions quickly and affordably.
By automatically grading news articles and social media posts as positive, negative or neutral, the sentiment analysis software can help public relations and marketing departments determine how well their campaigns are working by analyzing trends in consumer sentiment. Political campaigns can use sentiment analysis to study the public’s reaction to candidates’ positions; investors and stock analysts can analyze perceptions about the stock market or a particular stock.
Automated systems are less expensive than paying human analysts. However, they are not always the best choice. Although the software has improved, it remains far less accurate than human analysts. Automated programs cannot place comments in context and often fail to grasp nuances of human language. They have difficulty interpreting slang and sarcasm.
Pointing out an example of automated analysis can misfire, Huffington Post business blogger Dan Mirvish noted that when Anne Hathaway was in the news with favorable publicity, Warren Buffett’s Berkshire Hathaway’s shares went up. He found six dates going back to 2008 when that happened. Automated trading programs apparently confused the actress with Buffett’s company.
Questions to Consider
Katie Paine, CEO of Paine Publishing and a noted expert on media measurement, recommends questions to ask to help decide if sentiment analysis is right for your organization.
Recommended for YouWebcast: SEO in 2016: How Search Has Changed & Why Modern Marketers Must Adapt
- Do you receive more than 2,500 qualified mentions a month in social media? Qualified mentions do not include spam, content-farm generated copy or mentions of similar-sounding brand name. If your brand has relatively few mentions, a computer program might be more expensive than a trained analyst.
- Do people express any sentiment when discussing your brand? Conversations about some products, such as B2B products, are typically very factual. It may not be possible for a computer to glean any sentiment from the conversations.
- Do you have direct interaction with customers? If you have no direct customer interaction, then determining if sentiment has any real impact on your business will be difficult. You can link sentiment to customer behavior only if you are an online retailer or are in a field where people make reservations, or register online.
- What level of accuracy is acceptable to your executive leadership? Most automated tools have an accuracy rate of about 60% to 70%. If that is not acceptable to your leadership, you can use human coders to conduct a random sample check to ensure a higher degree of accuracy.
- What level of detail do you need from your sentiment analysis system? You will probably need a human analyst if you need to track complex messaging, quotes, issues, positioning, or other esoteric details.
- Do you run numerous campaigns, which will require different search terms and different message tracking and different definitions of positive or neutral? Computers take weeks to reprogram, test and fix. If you need fast turnaround on changes to your system, then use a human.
The Hybrid Approach
Some monitoring services combine automated sentiment analysis with evaluation by human analysts. By using human analysts, companies can establish finely-tuned criteria for what constitutes a positive or negative mention. Although not always perfect, human analysts are trained to recognize language nuances and idiosyncrasies that trip up automated programs. They can also be quickly trained on specific rating criteria for each client.
Humans are also less literal than machines and can better tease out meaning. Large companies with thousands of media mentions can the control costs of human analysis by reviewing a sampling of posts. Many companies, for instance, use human analysis only for media mentions that appear in top tier media sources.
PR measurement and data analysis experts agree that integrating automated tools and human analysts is the best approach for most organizations.
Experts at the Sentiment Analysis Symposium favored using a combination of software and humans.
“Machines do analytics, humans do analysis,” remarked Anjali Lai, an analyst at Forrester Research, according to Adweek.
Bottom Line: Automated sentiment programs provide both benefits and pitfalls. In many cases, human analysts provide superior results. It’s essential to carefully weigh the pros and cons before establishing a media monitoring and measurement plan. For many companies, a hybrid approach that integrates humans and machines is the most effective and cost-efficient option.
This post was originally published on the CyberAlert blog.
Is Automated Sentiment Analysis Right for Your Media Measurement?
Aucun commentaire:
Enregistrer un commentaire