“Opening the Black Box” is our ongoing series on the data treatments, assumptions, and methodologies behind our analyses. In this series, we answer frequently asked questions on how we build, validate, and make decisions about our data.
Thoughts or feedback for us? We welcome it.
Benchmarking global institutional investors presents a genuine challenge. Funds operate across multiple geographies, each with its own regulatory requirements and reporting standards. They each have unique ways of operating. Yet, we need apples-to-apples comparisons to develop credible and actionable insights.
To this end, CEM standardizes the data against our Global Reporting Principles, a set of principles developed and refined over more than 30 years of working with institutional investors around the world. This standardization is conducted on a best-efforts basis, and we assess the quality of each submission through our Data Scorecard to track both completeness and timeliness.
The data we receive is verified through a combination of intelligent automation and experienced human review. At the centre of this is CEM’s rules engine, an automated validation tool, working alongside a dedicated analyst assigned to your account. Together, they guide your data through a structured seven-step process.
How we hear from you
CEM receives feedback from subscribers through several channels. These include one-on-one conversations with CEM Relationship Managers, live feedback at our conferences, working groups and Steering Committees for specific research or product projects, and formal input through our Client Advisory Boards. Each of these channels serves a different purpose, and together they give us a well-rounded view of what our subscribers need.
Step 1: Data upload and pre-submission review
The majority of data is self-reported by funds. However, in some cases, CEM also sources certain data from third parties such as recordkeepers and asset managers, on your behalf. You get to review your data to correct any issues before submission. In the next blog, we will share what data can come from third parties.
Step 2: First-order review
As data is entered into our proprietary online platform, our system reviews the data in real time. If something appears unusual, such as a significant gap between your benchmark return and your actual return, the system immediately flags it for your attention. This gives you the opportunity to confirm or correct before proceeding.
Step 3: Second-order review
Once submitted, a second automated review checks for critical omissions and compares your data against your prior year’s reporting. These automated checks are conducted by CEM’s rules engine, a proprietary tool built from reviewing three decades of institutional reporting. The rules engine reviews data for 1) critical errors, which must be resolved before the submission can be accepted, and 2) non-critical anomalies, which require an explanation. The rules governing this engine are reviewed annually by CEM’s Data Governance Committee.
Step 4: CEM expert review
Your dedicated analyst then reviews the submission in detail, drawing on internal documentation of definitions to support consistent treatment across all clients. Complex or novel cases get escalated to senior experts or, in some cases, to the Data Governance Committee, so that every fund’s data is handled with the same rigour, regardless of its uniqueness or complexity.
Step 5: Client Q&A
Your analyst works directly with you through email or live call to confirm that data is interpreted accurately. Where data is missing or not directly comparable, an established imputation methodology is applied to maintain the integrity of the analysis. We will share more on our imputation methodology in a future blog.
Step 6: Data rejection
Data sets that are missing critical information or are not deemed fit for comparison are rejected. Rejected data does not become part of Combined CEM Investment and Administration Benchmarking Database CEM’s database. This ensures the robustness of the benchmarks you rely on. We will share more about the types of data we reject in a future blog.
Step 7: Data summary and joint review
A data summary is provided to you before issuing your draft results to ensure data is reflected correctly. Then, CEM’s relationship manager reviews the results with you in your draft report before your report is finalized.
This process reflects our commitment to accuracy, transparency, and a partnership approach.
Our committment to you
The trust our subscribers have placed in us over the years, many for close to a decade without interruption, is something we take seriously. This series is one way of honouring that trust. We are not here to simply explain ourselves. We are here to open a dialogue that pushes our work, and the industry, forward.
We look forward to hearing from you. If there is a question you would like us to address, please reach out to us at insights@explorecem.com.
This blog is part of our “Opening the Black Box” series. If you’d like to receive new posts in your inbox, please click here to subscribe.