AVS Methodology

[vc_row][vc_column][vc_column_text]

| Analyst Value Survey Methodology |

[/vc_column_text][/vc_column][/vc_row][vc_row][vc_column][vc_column_text]

EXECUTIVE SUMMARY

[/vc_column_text][vc_column_text]The Analyst Value Survey (AVS) is the only public survey into how thousands of professionals value different analyst firms like Gartner, Forrester and around 100 of their most successful competitors. The survey has been held since 2001 and the results are being read by an ever-increasing audience.

This document is your guide to understanding how we create the Analyst Value Survey and its deliverables. It outlines the methodology that the Analyst Observatory uses.[/vc_column_text][vc_column_text]

WHY THE ANALYST OBSERVATORY CONDUCTS THE ANALYST VALUE SURVEY

[/vc_column_text][vc_column_text]The Analyst Value Survey shows which industry analysts firms are most valued, and by whom. Since 2001, we’ve asked thousands of professionals about the analyst services they use. The results help you to understand which analysts firms have the most impact, and which will give you the most value. Each year, around 1,000 professionals complete the survey, with around 15% from Asia Pacific, and the rest evenly shared between the Americas and EMEA. 

The AVS offers three big benefits to clients: our detailed report, typically between 100 and 150 pages, a recorded webinar to give you the overview and a face-to-face video workshop where we help you use the AVS to develop your strategy.

The AVS is a unique evaluation of the IT Research marketplace based on two high-level criteria, use and value.[/vc_column_text][vc_tta_accordion][vc_tta_section i_icon_fontawesome=”fa fa-bookmark” add_icon=”true” title=”What is use?” tab_id=”1439977787174-5b70147f-a1ba”][vc_column_text]We ask professionals which analysts firms they use. Analyst research has changed a lot over the last 20 years, when analysts insight was consumed by reading expensive reports. Now analysts have a wide range of services, and have have a substantial impact on companies that don’t have formal subscription agreements. The AVS prompts users to comment on the 100 largest analyst firms, and also allows people to write in the names of other firms they work with.[/vc_column_text][/vc_tta_section][vc_tta_section i_icon_fontawesome=”fa fa-bookmark” add_icon=”true” title=”What is value?” tab_id=”1439977787306-acaf0e12-273e”][vc_column_text]It’s easy to compare the revenue of analyst firms, but the ability of those firms to capture value is not the same as their ability to generate value for users. Thousands of people tell us which services they use, from which analysts firms, and which services they value the most. [/vc_column_text][/vc_tta_section][vc_tta_section i_icon_fontawesome=”fa fa-bookmark” add_icon=”true” title=”What methodology is used?” tab_id=”1439977960976-a9ec9a14-db57″][vc_column_text]The AVS prompts users to comment on over 100 notable analyst firms, and also allows people to write in the names of other firms they work with. The firms are presented in two lists. The main list includes the 60 analyst firms most commented on in the previous years’ AVS, including every firm with the full-time equivalent of ten or more analysts. Each year we also have a candidate list: we ask participants which of 40 to 50 smaller firms they would have commented on if they had been in the main list. In the following year’s survey we relegate firms on the main list which used less than firms on the candidate list. In the 2016 survey, for example, Avasant moved up to the main list and Greyhound Research moved down to the candidate list.

The final report aims to show the data we collect as directly as possible. Most charts in the detailed report show actual percentages, rather than weighted scores. When we aggregate the data, the only weights come directly from the use and value scores given by respondents to the survey.

Responses are kept anonymous from both clients and the staff who produce the report. Staff use base data for report production that are free from personally-identifying variables (e.g. personal names, company names, email addresses, IP addresses and MAC addresses). In the reports, numerical responses are presented only as aggregate averages. Free text comments are described only by segment (for example, “Enterprise user, North America”). We do not publish any comments by respondents we feel might be personally identifiable.

While the survey does not prevent multiple responses from one firm, it does prevent multiple responses from the same machine.[/vc_column_text][/vc_tta_section][/vc_tta_accordion][vc_column_text]

THE AVS PROCESS ROAD MAP: HOW DOES THE AVS WORK?

[/vc_column_text][vc_column_text]The Analyst Value Survey process generally spans four key milestones, each of which has specific deliverables.

  1. The participants’ webinar.
  2. The detailed annual report
  3. Debate in our fora and workshops
  4. Analyst Firm Awards launch lunch and segment reports

[/vc_column_text][vc_column_text]

HOW DO I USE THE AVS?

[/vc_column_text][vc_column_text]The AVS is the first tool you need to understand which analyst firms to use, to hire or to pitch to.

  • Technology Buyers: Buyers of analyst services use the results of the AVS to see which firms are perceived as delivering the highest impact on their IT decision making.
  • Technology Providers: Analyst Relations and marketing professionals at technology providers use the AVS to improve and justify their prioritization of relationships with analysts who impact their business.
  • Research Firms: Analyst Firms use the AVS to understand how they can deliver more value and overtake their competition. As the M&A market grows, many analysts firms use the AVS to consider partnerships and investment opportunities.

    Keep in mind that high-value analysts might not be the only option for you. Smaller firms without a full portfolio of slick services can be cheaper, and sometimes just as close to growing market sectors. A small analyst firm might have a lot to offer and, indeed, many upstart firms are now delivering more value than the major analyst stalwarts.

[/vc_column_text][vc_column_text]

ANALYST OBSERVATORY OBJECTIVITY

[/vc_column_text][vc_column_text]For many professionals, the AVS is the compass used to help themselves and their stakeholders to navigate through the complex global market. That is why we aim to maximize the participation in the survey, in order to better assure its objectivity and resilience. We are committed to avoiding bias and potential conflicts of interest. The design, execution and analysis of the survey is separate from its commercialization. The Analyst Observatory’s researchers are committed to the highest standards of ethics in the research process.[/vc_column_text][vc_column_text]

QUESTION: AN ANALYST FIRM MENTIONED IN THE AVS SEND ME AN EMAIL ENCOURAGING ME TO PARTICIPATE. IS THIS ALLOWED?

[/vc_column_text][vc_column_text]Because there are no commercially-available databases of analysts’ users, the AVS relies on the Observatory’s extensive mailing list, and invitations which are shared by our staff, sponsors and partners. Several analyst firms also share links to the survey: firms are asked to use a custom URL which allows us to compare responses from different sources.

On the plus side, when firms share the link the survey gets much more data about them and their competitors. On the negative side, it produces some concerns that the results could become imbalanced. Our researchers have been running the survey since 2001 and take those concerns seriously. When we compare the responses, we see that respondents who come from invites from analyst firms are not notably different in their thoughts about those firms than those who do not. What it does do, however, is push up the share of voice of those firms and their competitors. For example, respondents who use the firms that have shared links to the survey are more likely to use IDC than are those who are not. So those firms are also boosting IDC’s profile, although there’s no notable difference in how those respondents feel about IDC.

After consulting with many people, including members of the Market Research Society, we have designed our survey accordingly.  We focus on average scores, so that changing the share of voice doesn’t impact on the outcomes.

Because of this design we believe that it is, on balance, a good thing that analyst firms help us gather more data.[/vc_column_text][/vc_column][/vc_row][vc_row][vc_column][vc_btn title=”Back to the main page” style=”custom” custom_background=”#eb6e31″ custom_text=”#ffffff” shape=”square” align=”center” i_icon_fontawesome=”fa fa-arrow-left” add_icon=”true” link=”url:%2F||”][/vc_column][/vc_row]