Why Evaluate?
Evaluation is a powerful tool that can scientifically prove how effective, essential, and profound your work is. The data collected from surveillance and evaluation (S&E) activities are the very foundation of the Wisconsin Commercial Tobacco Prevention and Treatment Movement. Evaluation can and should help us make better decisions and hold us accountable to goals set forth within the 2020-2025 TPCM State Plan. Your evaluation efforts will be able to inform and educate Wisconsin residents, policy-makers, and world-changers about the dangers of commercial tobacco use.
Data Equity
Adapted from We All Count's Foundations of Data Equity course.
Data-based evaluations are an important way to understand how our work is affecting the people in our communities. But how do we determine what constitutes an improvement? What is our yardstick for success? In many cases, impact evaluations look at improvements in income, health, or life expectancy. But what about equity in impact evaluations?
Unfortunately, equity in impact evaluation is often overlooked because we live and work in a system designed to place importance on those historically in power (white cis-males) and to ignore folks who don't fit in that box. Luckily, we have the ability to use evaluation to change the narrative and shine a light on those who are typically ignored.
It can be messy. It can be uncomfortable. Equitable data and evaluation is (relatively) a new field and so we are all learning as we go. To create a lasting, sustainable change, it is important that we work together and learn from each other, our successes, and mistakes. It means acknowledging gaps in our current system and admitting past injustices. It might seem daunting or overwhelming, but please remember that you have an entire movement full of people with experience supporting you as you go.
Scope of Work Resources
Coalition Member Survey:
The purpose of this activity is to ensure that coalition members have their voices heard and their needs reflected in the coalition’s work. The survey should be a quick temperature check of coalition member engagement, how confident coalition members are in the coalition’s work, and help serve as a guide for future coalition efforts.
Key Informant Interviews: In order to authentically engage with your population of focus, you need to create a mutually beneficial relationship with leaders and members of that community. These interviews are designed to help you listen and learn about and with those most deeply involved in the work. This activity can be adjusted based on the relationship between the coalition and population of focus to best fit your needs. Key Informant interviews can help your coalition prioritize future goals, campaigns, and partnerships.
Public Opinion Surveys: Public opinion polls will help you determine where your community stands on a certain issue. This can help inform your coalition about targets for educational advocacy, where to organize your next event, how to talk about tobacco use, and so-on and so-forth. It is important to clearly define your outcome goal before you begin the survey process so you can determine the best audience to complete the survey and what questions should be included in the survey..
- Potential Survey Topics (not exhaustive)
- Public Opinions of Policies
- Public Opinion of Tobacco Use in Community
- Health Behaviors/Habits
- Community support/opposition to Laws
- Perceived harmfulness of tobacco products
- Example Surveys
- Link to the 2020-2025 TPCM State Plan
- Question banks
- Surveillance Methods
- Survey platforms
- Survey Monkey
- Google Forms
- Qualtrics
- Survey Gizmo
- Old fashioned Paper and Pencil
- Resources for Surveillance and Evaluation
- Qualitative vs Quantitative
- Qualitative data asks how something happened—it includes descriptions, characteristics, and subjective interpretation. To read more about qualitative data, click here.
- Quantitative data is data that can be given a number and counted. The common statistics used in tobacco prevention and control, like use prevalence, is quantitative data.
- Process vs Outcome Evaluation
- Process Evaluations: Process evaluations measure how a program's impact was achieved. These evaluations are usually short and monitor the implementation of activities (e.g. media, social media, trainings, meetings, presentations, etc.).Learn how to develop process evaluation questions.
- Outcome Evaluations: These measure the impact of a program. These are usually longer than process evaluations and are used to monitor the effectiveness of a program. They are easy to do and can provide important feedback. One way to conduct outcome evaluations is through pre- and post-tests to evaluate change in knowledge through workshops, trainings, and presentations using the same questions in both tests, one before and one after the activity.
- Changing the narrative
- Webinar links
- Helpful external resources:
- Data Viz blogs:
- Data and design:
- https://datastudio.google.com/navigation/reporting
- https://www.cdc.gov/nceh/clearwriting/docs/clear-writing-guide-508.pdf
- https://editor.method.ac
- ttps://www.columnfivemedia.com/how-to-find-stories-in-data
- http://vanseodesign.com/web-design/gestalt-principles-of-perception/
- https://www.smashingmagazine.com/2014/03/design-principles-visual-perception-and-the-principles-of-gestalt/
- Data and colors:
- https://mycolor.space/?hex=%23845EC2&sub=1
- http://tristen.ca/hcl-picker/#/hlc/6/1.05/B7644A/B15D4F
- https://maketintsandshades.com/
- https://gka.github.io/palettes/#/9|s|00429d,96ffea,ffffe0|ffffe0,ff005e,93003a|1|1
- https://www.color-blindness.com/coblis-color-blindness-simulator/
- https://abc.useallfive.com/
- https://contrastchecker.com/
- https://carto.com/carto-colors/
All reports and infographics
- Tobacco Facts: Consumption, Mortality, and Morbidity (March 2019)
- Tobacco Facts: Youth (February 2019)
- Tobacco Facts: Adults (November 2018)
- Tobacco Facts: Adults (November 2019)
- Tobacco Facts: Adults (February 2021)
- Tobacco Facts: Smoking During Pregnancy (May 2019)
- Up Close with Tobacco and…
- Helpful evaluation websites:
What is it?
- This is a Tobacco Prevention and Control Program led work group focused on evaluation. This learning group is created to provide a time and space for movement members to come together to learn, teach, review, and ask questions about surveillance, evaluation, and data visualization.
- If you’d like to join, please email Emile Gunovich.
Anticipated Projects
- Reviewing the 2020-2025 TPCM State Plan
- Create the new 2020-2025 TPCP Evaluation Plan to complement the State Plan
- Evaluation bi-monthly newsletters
Remember to favorite and visit the Evaluation Google Drive often!