This website is a good resource for individuals and community groups working for change in their communities, specifically with issues relating to race and racial equity. The site is written for users who are new to evaluation -or perhaps those who give funds related to racial equity, but who are not yet clear on how to evaluate it. The site discusses how to apply a “racialized” perspective to evaluation, meaning using the ideas of racism, oppression, privilege and access to power as a lens through which evaluation questions are developed and results are analyzed.
Resources
The Annotated Guide to Tools & Resources provides tools, frameworks, and resources to help you develop and implement your evaluation. It’s a repository of useful, practical materials that can help you create an evaluation plan; design your evaluation approach; develop or adapt tools and instruments; and otherwise move your evaluation forward.
This Guide was originally assembled from many sources and fields and annotated by evaluator Suzanne Callahan of Callahan Consulting for the Arts. We continue to add resources. Your suggestions are welcome!
Do you have a useful tool or resource to add? Contact animatingdemocracy@artsusa.org.
Publication Date: May 23, 2017
Resource Format: practical tool
The Evaluator / Researcher Companion guide to Animating Democracy’s framework, Aesthetic Perspectives: Attributes of Excellence in Arts for Change, offers ideas and insights to help evaluators and researchers apply the framework to address their needs and interests. Aesthetic Perspectives aims to enhance understanding and evaluation of creative work at the intersection of arts and community/civic engagement, community development, and justice.
Publication Date: July 1, 2014
Resource Format: book / article
Commissioned by the Working Group on Philanthropy for Social Justice and Peace (PSJP), Framing the Discourse, Advancing the Work, by Moukhtar Kocache, speaks to funders as an exploration of philanthropy at the nexus of social justice, peace, and arts and culture. It presents a broad overview of the relationship between progressive social change work and the arts and explores the role of philanthropy in supporting this work. It sets out recommendations for how philanthropy might further its engagement with work at this nexus.
Publication Date: May 23, 2017
Resource Format: practical tool
In the Funder Companion to Animating Democracy’s framework, Aesthetic Perspectives: Attributes of Excellence in Arts for Change, author Chris Dwyer of RMC Research offers ideas and insights to help funders apply the framework to address their needs and interests. Aesthetic Perspectives aims to enhance understanding and evaluation of creative work at the intersection of arts and community/civic engagement, community development, and justice.
To get insight into a complex community, problem or process of change, sometimes you need to look beyond conventional research or evaluation methods. Ethnography is a powerful way to step inside the culture of an organization or community, hear ongoing feedback from multiple points of view, and understand a program's real impact. This guide describes ethnography's benefits and pitfalls, and how the method is used to document, evaluate and improve approaches to youth engagement, HIV education and neighborhood policing.
Publication Date: June 30, 2014
Resource Format: practical tool
Created by the Georgia Council for the Arts, this guide provides a useful entry for local and state arts agencies and arts organizations newly interested in evaluation. The guide is designed not only as an introduction to evaluation, but to demonstrate how evaluation can be used as an internal learning tool within an organization to help manage programs, spend funds wisely, and assist constituents and potential investors to better understand their work.
Resource Format: practical tool
An outstanding piece, this 51-page handbook presents a great discussion of outcomes as they relate to policy and advocacy. Intended for grantmakers but useful for nonprofits as well, it encourages users to think about measurement of advocacy and policy efforts toward social change. It offers outcomes lists, examples of data collection tools, and many sample instruments from which users may benefit.
Publication Date: January 22, 2016
Resource Format: book / article
Continuum: Attitudes, motivation (values, aspirations) | Awareness, knowledge, understanding | Behavior, participation (action, activism, advocacy) | Systems, policies, places (legislation, issue/problem resolution)
Animating Democracy held its second Evaluators Circle (E-Circle) at the Japanese American Cultural & Community Center, inviting Los Angeles-based cultural leaders, researchers, and evaluators to share evaluation projects, findings and learning. It was a rich and hearty exchange.
Publication Date: February 1, 2011
Resource Format: practical tool
FSG and Knight Foundation’s IMPACT: A Practical Guide to Evaluating Community Information Projects aims to help organizations collect useful information about the impact of their community information projects. Community information projects incorporate media, news, and online and digital communication to build knowledge and engagement capacities in community.
Resource Format: book / article, practical tool
Written for evaluators, this 17-page report is a very nice guide to communicating about culture in evaluation practices and may be used as a reference in data collection. The report emphasizes that cross-cultural competency is a necessary skill for evaluators to have. The author encourages evaluators to think of culture as a factor to be considered as much as sampling and measurement. In addition to the introduction and conclusion, the paper has three main sections: culture, social identity and group membership, and privilege and power.
This 33-page document calls itself a study but can serve as a guide to indicator development. Though done in 1996, it an extremely thorough, thoughtful overview of indicator consideration and development. It defines indicators, gives examples of types, and talks of the challenges of measuring difficult areas such as political development. It discusses issues related to indicators in governmental agencies versus NGOs.
Publication Date: June 30, 2015
Resource Format: case study
Featuring Alternate ROOTS, Jane Addams Hull-House Museum, and The Theater Offensive, this rich and rigorous publication examines the contours, possibilities and limitations of adaptive change for three arts and social justice organizations participating in EmcArts Innovation Labs for the Arts. These organizations have social justice missions and mandates, and are deeply connected to histories of social movements in their respective communities.
From the Centers for Disease Control and Prevention, this 71-page report defines process evaluation and describes the rationale, benefits, key data collection components, and program evaluation management procedures. Within the framework of discussing tobacco use prevention, this paper is a very good primer for process evaluation for readers in a variety of fields. It provides clear and well-presented charts, graphics, principles, and summaries to help guide readers.
Resource Format: book / article
Continuum: Awareness, knowledge, understanding
Understanding how knowledge moves through communities can help evaluators select the components of a program that are best suited for scaling up. “Scaling” is the practice of replicating programs for implementation at new locations, by other organizations, or to serve larger groups of people. Knowledge-for-action theories address how learned information becomes the measureable impact of arts for change work. This article highlights five theories of knowledge transfer: utilization, diffusion, implementation, transfer, and translation.
Publication Date: August 31, 2008
Resource Format: practical tool
This set of resources is useful to organizations that want to increase their social impact. The Kellogg Foundation asks, “Ever wonder if you are getting through to people?” They offer twelve principles in the Knowledge to Understanding framework that aid in knowledge transfer. An introductory video provides a snapshot of the framework and a report and interactive slideshow provides more depth into all twelve guiding principles.
Resource Format: book / article, case study
This 35-page report provides a summary of findings and lessons learned by the Neighborhood and Family Initiative (NFI), a comprehensive community initiative launched by the Ford Foundation that began in 1990. NFI was a 10-year effort that sought to strengthen a single neighborhood in each of four cities and to improve the quality of life for the families who live in them. The report is divided into two sections; the first provides a brief overview of NFI while the second section distills the lessons learned by NFI over the course of implementation.
Publication Date: January 1, 2013
Resource Format: website
In Making Data Visualizations: A Survival Guide and Other Resources, Beth Kanter, author of Measuring the Networked Nonprofit, blogs about the importance of data visualization strategies in the nonprofit sector. In this list, Kanter offers her own take on six resources that provide an entry point into the field of visualization. These resources range in length from full books on graphics to brief tips on creating effective yet simple presentations. Below, Animating Democracy notes resources that we believe to be the most useful:
Publication Date: July 1, 2014
Resource Format: book / article
Patterns of participation for arts and cultural activities have resisted significant change. In Making Meaningful Connections, The James Irvine Foundation researches attendance patterns of small, community-based cultural organizations and events. Attendance diversification is not only important to organizations that produce “benchmark arts activities,” but for smaller organizations that may work within specific demographic populations and would simultaneously like to expand their community participant base.
This piece is an excellent primer on how to think about outcomes and the hurdles that may arise in measuring them. It addresses and deals with challenges such as measuring intangibles. Published by GrantCraft (a division of the Ford Foundation), the eleven-page guide is written for grantmakers to describe outcomes-based evaluation. It defines key terms and makes a case for why outcome measurement is important.
Resource Format: book / article, case study
Los Angeles Poverty Department (LAPD) is a Skid Row-based theater organization, founded and directed by artist John Malpede. LAPD has distinguished itself by its longstanding commitment to making change in L.A.’s Skid Row community, particularly regarding the homeless, through theater-based civic engagement work. Many have observed LAPD’s apparent potent effects on individuals and on social relations in Skid Row, and acknowledge its contributions to influencing structures, systems, and even policy.
"What are we doing, and why do we think it’s going to make a difference? Are we being effective?" Grantmakers ask evaluation questions like these of their grantees and themselves. This brief guide explains why grantmakers use theories of change to guide their questioning, unearth assumptions that underlie their work, establish common language, and develop strong action plans. Contributors to the guide also describe how a theory of change sets the stage for evaluation by clarifying goals, strategies, and milestones.
Written for funders, this 43-page piece is about social entrepreneurship, its relatively new place in the field of philanthropy, and implications for evaluation. Developed from research with grantees and two dozen interviews with funders, it talks about flexibility, leadership, and risk in funding innovation, and why these are key.
Publication Date: February 1, 2014
Resource Format: book / article, practical tool
This report provides a comprehensive overview and assessment of the approaches that are currently employed in impact assessment of media as a socially valuable tool. “Social value,” in this context, is described in terms of outcomes that extend beyond financial measures of success such as improving the well-being of individuals and communities. This report seeks to identify relevant analytical approaches, methodologies, and metrics for assessing media’s social impact that can inform further work in this area and that appear particularly promising.
Resource Format: book / article, practical tool
This article from the Social Edge website (a program of the Skoll Foundation) aims to connect practitioners of the social benefit sector to network, learn, and share resources. It provides useful language and concepts about measuring outcomes in the social science field. Written to "social entrepreneurs," the piece explains that qualitative information can be effective in measuring social change, but one needs to be systematic in collecting and using qualitative data to evaluate programs.
Publication Date: October 9, 2014
Resource Format: book / article
Measuring the Networked Nonprofit offers tools and strategies to nonprofits searching for reliable and measurable data from their social media efforts. This book is a hands-on resource for nonprofit professionals who must be able to accurately measure the results of their social media ventures.