Research & Evaluation
Click2SciencePD conducts evaluation and research studies to determine the effectiveness of our blended professional development model, training interventions, and resources. The brief synopsis below outlines the details of our efforts and opportunities to participate in validating a tool to measure staff self-confidence. For more information on any of the evaluation and research projects listed below, please contact us.
Beyond School Bells (BSB)
Interviews with Coach/Trainers
Five participants involved in out-of-school time (OST) activities in Nebraska were interview by phone or email. Participants were selected for participation because Click2SciencePD provided training for their staff and site directors. Individuals identified themselves broadly as professional development (PD) coach/trainers.
Participants were asked a series of questions regarding the PD activities in their program (i.e., what PD is being done, who is providing PD, how are you providing PD) and barriers to providing PD. For instance, participants were asked to reflect on Click2Science and the effectiveness of the PD trainings they received in Click2Science (i.e., changes or improvements noticed with staff and program quality). The intent of these interviews was gather feedback about the effectiveness of the PD intervention and to learn more about PD opportunities in Nebraska.
University of Minnesota Extension
The University of Minnesota Extension conducted an evaluation of Click2Science in July 2014. Two methods, a think aloud protocol (TAP) and new user surveys, were used to understand OST directors, trainers, and site supervisors’ perceptions of the utility and quality of Click2Science. The two pilot TAP activities include a small, targeted group of Click2Science users. During the TAP pilots, evaluators took notes on participants’ observations (think out louds) and watched them navigate the website resources. Evaluators led the TAP pilots using distance technology (i.e., AdobeConnect). In addition to the TAP pilots, individuals completed new user surveys that included both open and closed-ended questions about Click resources (N=14).
Nebraska Academy for Methodology, Analytics and Psychometrics (MAP Academy)
Multiple-Case Study of Click2Science Early Adopters
This report documents the main themes that emerged from interviews with individuals involved in out-of-school time (OST) educational activities across four states. Ten interviews were conducted using either phone or video-conferencing technology (e.g., Skype). Participants were included if they were an early adopter/participant with Click2Science. Individuals identified themselves broadly as network or site-level representatives and professional development coach/trainers. The report provides a summary of the context of each location and common themes found in participants’ experiences with Click2Science across the four states. Participants provided information on the OST context in their particular location and their use of Click2Science resources.
General feedback from the early adopters highlighted the flexibility, utility and applicability of Click2Science materials for their OST programs.
Comprehensive Triangulated Evaluation
Researchers from the MAP Academy completed a comprehensive triangulated evaluation of Click2Science, which included interviews from site leaders and frontline staff, pre/post observations of frontline staff using the Dimensions of Success (DoS) observation tool, and youth survey data. Four YMCA locations participated in five hours of training activities using Click2Science and Better Kid Care materials. View the Triangulation Report complete results from this evaluation, or check out our one-page infographic for an evaluation overview.
Researchers from the MAP Academy are currently testing self-confidence surveys for the Click2Science skills. The retrospective post-pre surveys ask frontline staff/volunteers to rate their confidence levels on several items using a Likert-type scale. Participants first rate their confidence level after a professional development experience using Click2Science resources and then rate their confidence levels before the professional development experience. Following these scales, there are open-ended questions, demographic and program information.
If you are interested in assessing your staff’s self-confidence levels following a professional development experience facilitated with staff development guides or other materials from Click2Science, please email firstname.lastname@example.org for a coach/trainer code. Please indicate if and when you would like the data shared back with you.
Ask your staff to click the link below to be taken to the survey, where they will consent to participating in the research, select the skill they’ve focused on, complete the survey, and input your coach/trainer code.