“Opening the Black Box” is our ongoing series on the data treatments, assumptions, and methodologies behind our analyses. In this series, we answer frequently asked questions on how we build, validate, and make decisions about our data.
Thoughts or feedback for us? We welcome it.
As a data and insights firm serving institutional investors globally, data integrity underpins all the work we do at CEM. We govern our data through eight foundational principles owned by CEM’s Data Governance Committee, whose mandate and membership we will cover in an upcoming blog. Each principle addresses a distinct dimension of how we work with data, and together they define how we handle every data point that flows through our systems, from collection to reporting.
1. Completeness
We hold a high bar for valid and actionable data. Our preference is that data is provided directly by our subscribers. We recognize that is not always possible, so where data sets are incomplete, we follow a clear and consistent approach.
For data to be included in the CEM database, it needs to be unambiguous. We need to be able to interpret what has been provided with confidence, which requires a clear separation between data that has been reported and data that is genuinely missing. We treat incomplete data as missing, not as zero. Where appropriate, we impute data using standardized methodologies. We will share our standardization methodologies in more detail in upcoming blogs.
Two examples help illustrate this. With respect to investments, if you do not provide underlying investment management fees for your private equity fund of funds, we treat those as missing and impute them from our database so your program’s costs remain comparable to peers. If there are legacy funds that are no longer charging fees, we want to ensure that fees are reported as zero and not unknown.
On the administration side, if you do not provide data for key service metrics, e.g., total logins to the secure member area, or undesired call outcomes, we impute it based on the historical and peer results in our pension administration database.
2. Comparability
All data and insights are presented in a consistent context across every key dimension of comparison. Where unavoidable differences exist, they are clearly flagged. We maintain a detailed guide of definitions for every survey data point. The definition guide is accessible through our online survey platform. This ensures that your data and your peers’ data are handled in the same manner, which is the foundation of meaningful benchmarking.
If a certain data point is not comparable, we call it out in your report through clear commentary. Certain costs are deliberately excluded from benchmarking where consistent, comparable data is not available across the peer group, for example, costs like marketing expenses and health insurance that vary too widely to support a fair comparison.
You can learn more about what costs are included in CEM’s investment and administration benchmarking in a blog we published earlier here.
3. Accuracy
Every data point is validated through our proprietary tools and expert judgment. Your data is first reviewed by our automated rules engine, a proprietary tool that compares your current data against prior years. An analyst assigned to your account then reviews your data using our in-house tools, comparing it against your peers and our broader universe. Where data does not meet our quality threshold, entire data sets can be rejected. In the coming weeks, we will cover our full seven-step validation process.
4. Confidentiality
Individual, non-public data cannot be read or derived from any CEM materials. Outside of your own reports, your data is always reflected in statistical aggregates such as percentiles, medians, and averages. De-anonymized reporting is only produced with expressed consent and is made available exclusively to the closed group where all participants have mutually agreed to share their identity.
5. Providence
CEM retains the final say over all data-related decisions. We welcome all input and feedback from our subscribers. That includes feedback on our products, interest in specific peer group compositions, and reporting preferences such as timing or the treatment of carried interest in private markets comparisons. In an upcoming blog, we will discuss how subscribers provide feedback in more detail. The final decision, however, is always made in line with our principles and requires explicit approval from the Data Governance Committee or its delegates, following the framework that will be described in our next blog.
6. Timeliness
We provide insights on a timely basis, commensurate with the provision of data from a client and its peers. Delivery dates are agreed upon during onboarding meetings with new subscribers and survey kick-off meetings with existing subscribers. We communicate relevant delivery dates clearly and take all reasonable steps to meet them.
7. Transparency
Wherever data has been enhanced or imputed, we disclose it fully, including the methods used and the sources drawn upon. Any imputations or standardizations applied to your data are transparently disclosed in your reports.
8. Security
We apply industry-standard protections to all data and respond to any incident with speed, discipline, and open communication. In the event of an incident, we follow a documented, predetermined protocol that is revisited annually.
Our committment to you
The trust our subscribers have placed in us over the years, many for close to a decade without interruption, is something we take seriously. This series is one way of honouring that trust.
We look forward to hearing from you. If there is a question you would like us to address, please reach out to us at insights@explorecem.com.
This blog is part of our “Opening the Black Box” series. If you’d like to receive new posts in your inbox, please click here to subscribe.