Data Maturity Framework Questionnaire

Scorecard Category Question

Problem Definition
What is the problem you are trying to solve? What does success look like/how
Problem Definition much does the needle need to move?
Interventions What interventions do you have available to solve the problem?
If this is successful, what impact will this project have? Will it encourage future
Impact projects/goodwill?
Available Data What data sets do you have access to relevant to the problem?
Data Fields What fields are in each of the data sources? (See Data Sources Worksheet)
Size How many people/addresses/facilities/entities does the data contain?
For this problem, what % of entities are at risk or have resources to be
Target Population intervened?

Data Governance
For the data sets that you have access to - do you own the data? Do you have
permission to use the data? If you do not own the data, do you have the
Ownership relationships with the data owner?
Physical Accessibility Is the data accessible outside the department/agency? Is there a VPN?
What security policies and considerations need to be in place for each of the
Security Policy data sources? (HIPPA, FERPA)

Implementation and Maintenance

Technical Implementation Do you have people in house who can implement/deploy the solution?
Do you have the internal tech and data infrstructure to provide a continuous data
feed from all the systems, and integrate the results/recommendations back in to
Data Infrastructure the agency systems?
Maintenance Can you update, maintain, and support the implemented solution?

Data Readiness
Lagging Basic Advanced Leading
Accessibility How accessible is the data that's required?
Storage How is the data stored?
Integration How intergrated are the different data sources?
Relevance and Sufficiency Do you have data that is both relevant and sufficient to solve the problem?
Quality How is the data quality?
Collection Frequency How often is the data collected?
Granularity What is the level of granularity for the data sources?
History How much history is stored and how are updates handled?
Privacy What data privacy policies do you have in place?
Documentation How well documented are the data?

ORGANIZATIONAL READINESS
Lagging Basic Advanced Leading
How bought in are staff throughout the organization? What percentage of the
Staff Buy In staff are involved in data collection? Data analysis?
How bought in are the people on the ground doing the data collection? Do they
understand the importance and nuance of data collection? Do they get direct
Data Collector Buy In benefit from collection data?
How does leadership value data? Do they require data to be presented in order
Leadership Buy In to make decisions?
People Resources Do the people who will act on the results buy in?
Are there policies in place around who can use data, how they can use data,
Data Use Policy which parts can they use, and for what purposes?
Intervenor Buy In Do the people who will act on the results buy in?
How do your funders consider data? What kind of data do they require? What
Funder Buy In support for technology and personnel do they give you?
Data Maturity Framework
Data and Tech Readiness Scorecard

Category Area Lagging Basic Advanced Leading

Only accessible within the application Can be accessible outside the application All machine readable in standard open All machine readable in
Accessibility but proprietary format, requiring standard open format and
where it is collected specialized analysis software format (CSV, JSON, XML, database) available through an API
How is Data Stored

Storage Paper PDFs or Images Text Files Databases

Data is exported occasionally and Central data warehouse - realtime

Integration Data sits in the source systems External data also integrated
integrated in ad hoc manner aggregation and linking (Automatic)

The data you are collecting on subjects You have data that is helpful and relevant You have all the relevant data
of interest is irrelevant to the problem Some of the data you have is relevant, for solving the problem but not sufficient to about all the entities being
Relevance and Sufficiency you want to solve: ie you want to do but it is insufficient because key fields are solve it well. ie you have yearly academic analyzed and it's sufficient to
predict which students need extra missing, ie no data on academic behavior and demographic information but are
support to graduate on-time but don't or attendance history, etc. missing extra-curricular activities, or solve the problem you are
have data on graduation outcomes interventions they were targeted with tackling

What is Collected? Quality Missing rows (people/address level Missing columns (variables missing) No missing data but errors in data No missing data and no errors
entities missing in the data) collection such as typos in data collection
Collection Frequency Once and never again yearly frequently realtime
Individual level (person or address) level
Granularity City level aggregates Zipcode/Block level aggregates Incident/Event level data
data

Historical data is stored and new data gets All history is kept and new data
Historical data is stored but updates schema gets mapped to old
History No History Kept - old data is deleted overwrite existing data appended with timestamp, preserving old schema so older data can be
values used

Software defined/controlled
ad-hoc approval process in place that privacy protection that allows
Privacy No privacy policy in place no PII can be used for anything allows selected PII data to be used for analytics to be done while
selected/approved projects preserving privacy based on
predefined policies
Other
data dictionary plus full
no digital documentation or metadata: data dictionary exists (variables and data dictionary plus full metadata available metadata available including
Documentation data exists but field descriptions or categories defined) (including conditions under which the data collection assumptions, what's
coded variables are not documented were captured) not collected, and potential
biases
Data Maturity Framework
Organizational Readiness Scorecard
Area Lagging Basic Advanced Leading

Staff at the organization have some idea There are a few individuals who deeply Organization has a clear idea of how data Organization has a culture of data
Staff Buy In that data exists but doesn't understand it is understand the data available and what can be used to drive business decisions within the organization and demands
data to justify all programmatic
important can be done with it beyond justification of funding decisions
On the ground staff provide data in
real time and make decisions based
On the ground staff provide data seldomly, On the ground staff provide data on a on the data and insights available to
sporadically, or incompletely because they On the ground staff regularly provide data
Data Collector Buy In are required to but it is seen as a hindrance because they are required to regular basis and eventually get actionable them, and offer suggestions on what
to their "real job" insights in return is collected/what information they
could use to improve their job
effectiveness

Leaders at this level fundamentally don't Leadership has a clear idea of how data Leadership builds a culture of data
Leadership Buy In know how data can help advance the Leadership wants to use data but don't can be used to drive business decisions within the organization and demands
have a clear path forward to use data data to justify all programmatic
organization's mission. beyond justification of funding decisions
The organization knows how data can help, Organizations know how data can help,
what data they need, and are able to what data they need, and are able to The organization has dedicated staff
People Resources Individual stakeholders maintain siloed access it, but lack the in-house data skills, access it, but lack either the infrastructure who own data storage AND data
data sets tools, or infrastructure to be able to turn or the people to be able to turn data into content owners who own the cleaning
data into meaningful insights that affect meaningful insights that affect human and rigor of the data
human action. action.
Organization has policies in place for the Organization has policies in place for
Data Use Policy No policies exist around use, transfer, and use, transfer, and sharing of data but it Organization has policies in place for the the use, transfer, and sharing of data
sharing of data does not cover all data that exists within use, transfer, and sharing of data internally internally and externally
the organization
Partnerships exist and have policies and Partnerships exist and have policies
Intervenor Buy In No partnerships exist Partnerships exist but data is not shared technology in place to share data and technology in place to share data
occasionally or through a manual process in real-time
Funders require data driven decision
Funders do not require data other than Funders ask for key performance metrics making and provide funding for data
Funder Buy In vanity metrics Funders ask for key performance metrics and provide funding for data infrastructure infrastructure, maintenance, and
and maintenance usage