Senior Evaluator: Social Work Office of Research & Public Service - UTK

The Senior Evaluator provides day-to-day scientific and operational leadership for the three-year Supporting Families That Support Their Kin project, a statewide randomized controlled trial and implementation and cost evaluation of Tennessee's Relative Caregiver Program (RCP). Reporting to the Applied Research Team Lead and working with the Principal Investigator, Tennessee DCS, seven RCP providers, the Children's Bureau, and the federal Evaluation Technical Assistance provider, the Senior Evaluator leads the power analysis and final evaluation protocol; measure selection; recruitment, randomization, consent, retention, and analysis procedures; implementation, fidelity, qualitative, and cost studies; and integration of administrative, survey, provider, qualitative, and cost evidence. The position safeguards human subjects and study integrity, directs evaluation workflows, supports continuous quality improvement, and leads semiannual and final reporting suitable for federal review and potential Title IV-E Prevention Services Clearinghouse submission.

This is a grant-funded position and is contingent upon the continued funding of the grant. 

Responsibilities

Evaluation Design, Governance, and Management: 

• Lead development, federal technical-assistance review, revision, and implementation of the Children's Bureau-approved evaluation plan. 
• Finalize the theory of change, research questions, power and minimum-detectable-effect analysis, intervention and comparison conditions, eligibility criteria, measures, family-level randomization, consent and retention procedures, urgent-risk safeguards, administrative-data linkage, data security, and prespecified analysis plan. 
• Coordinate IRB materials, data-sharing agreements, protocol amendments, study registration and documentation, evaluation timelines, and compliance with PI-18-11 and Clearinghouse standards.
• Provide day-to-day work direction to the Data Analyst and other evaluation contributors.

Impact Study Leadership and Analysis: 

• Direct statewide RCT enrollment and centralized randomization monitoring across all seven providers.
• Oversee common baseline and follow-up measures, survey retention, DCS administrative-data linkage, assignment balance, crossovers, emergency overrides, protocol deviations, attrition, and missing data. 
• Lead reproducible intention-to-treat and approved secondary analyses of service connection, caregiver economic and housing stability, caregiver mental and emotional health, referrals, satisfaction, family functioning, child safety, permanency, and well-being. 
• Interpret uncertainty, sensitivity analyses, and practical significance without releasing premature causal findings.

Implementation, Fidelity, and Qualitative Studies: 

• Develop and apply the implementation framework, fidelity standards, dosage measures, provider-assessment procedures, interview/focus-group protocols, and adaptation documentation. 
• Evaluate reach, timeliness, service initiation, referral and linkage processes, service duration and intensity, provider and regional variation, rural access, facilitators, barriers, and deviations from the approved model. 
• Lead sampling, data collection, analysis, and integration for eight caregiver focus groups across immediate-service and delayed-access conditions and ensure qualitative procedures comply with the approved IRB protocol.

Cost Study, Performance Monitoring, and CQI:

• Specify cost-study methods and oversee collection and analysis of personnel, operating, stipend/service, provider, and implementation costs. 
• Estimate total costs, cost per caregiver family and child, variation by family size and service intensity, and relevant comparisons with foster-care costs. 
• Establish performance indicators and interpretation rules; review monthly dashboards, quarterly fidelity/data-quality findings, and exception reports; facilitate action-oriented CQI discussions; and ensure corrective actions have responsible owners, deadlines, and documented follow-through without compromising the experimental design.

Reporting, Dissemination, and Sustainability:

• Lead semiannual implementation and evaluation reports and the integrated final evaluation report; ensure conclusions are supported by approved analyses and clearly distinguish descriptive, qualitative, and causal findings. 
• Produce or review executive and plain-language summaries, presentations, provider-facing CQI products, methods and analysis documentation, and sustainability-oriented products. 
• Present findings to federal, state, provider, caregiver, policy, and professional audiences and assemble a submission-ready evidence package for potential Clearinghouse review after required Children's Bureau review.

Qualifications

Required Qualifications

• Education and/or Experience: 
  • Master’s degree in program evaluation, social work, public health, psychology, sociology, education, public policy, statistics, economics, or a related field. 
  • At least five years of progressively responsible professional experience conducting program evaluation or applied research beyond coursework, including evaluation design and management, quantitative and qualitative methods, statistical analysis, human-subjects or data-governance requirements, and preparation of reports for funders or decision-makers.
  • Experience must include responsibility for coordinating complex deliverables and clearly communicating methods and findings. 
  • A relevant doctoral degree with substantial applied evaluation leadership may substitute for up to two years of required experience.
• Knowledge, Skills, Abilities Knowledge, Skills, Abilities: 
  • Knowledge of rigorous program evaluation, experimental and quasi-experimental design, statistical power, causal inference, survey methodology, and mixed-method research.
  • Knowledge of implementation science, fidelity and dosage assessment, qualitative research, cost analysis, performance measurement, and continuous quality improvement. 
  • Knowledge of human-subjects protections, IRB processes, data-use agreements, confidentiality, secure administrative-data linkage, disclosure review, and ethical research practice. 
  • Advanced skill developing evaluation protocols, research questions, logic models, measures, sampling plans, analysis plans, and federal evaluation reports. 
  • Advanced skill applying statistical methods in R, Python, SAS, Stata, or comparable software and interpreting effect estimates, uncertainty, attrition, missing data, and sensitivity analyses.
  • Skill designing and integrating quantitative, qualitative, implementation, fidelity, and cost evidence. 
  • Skill managing complex, multi-partner evaluation workplans, risks, deliverables, and documentation within fixed deadlines. 
  • Skill communicating technical findings accurately and accessibly in written, oral, and visual formats for technical and nontechnical audiences. 
  • Ability to lead collaborative evaluation activities and provide work direction to analysts, researchers, students, consultants, and data-collection staff. 
  • Ability to exercise independent scientific judgment, protect study integrity, resolve complex methodological issues, and work effectively with federal and state agencies, providers, caregivers, and community stakeholders.

Applicants must be legally authorized to work in the United States on a full-time basis without need now or in the future for sponsorship for employment-based visa status.

Preferred Qualifications

• Education: 

  • Doctoral degree in program evaluation, social work, public health, psychology, sociology, education, public policy, statistics, economics, or a related field.

• Experience: 

  • Experience leading randomized or strong quasi-experimental impact studies and complementary implementation, fidelity, mixed-method, or cost evaluations; evaluating child welfare, kinship, or other federally funded human-services programs; working with longitudinal administrative and survey data; supervising or directing analytic work; collaborating with government agencies and community providers; and preparing evidence products for federal review or peer-reviewed publication.

• Knowledge, Skills, Abilities: 

  • Knowledge of child welfare, kinship care, Title IV-E, PI-18-11, Prevention Services Clearinghouse standards, or evaluation of federally funded human-services programs.

  • Knowledge of waitlist-control trials, longitudinal child welfare administrative data, service-access outcomes, family well-being measures, and community-engaged research. 

  • Skill leading multi-site or statewide impact, implementation, mixed-method, and cost evaluations from planning through dissemination. 

  • Skill preparing peer-reviewed publications, federal evidence submissions, preregistrations, reproducible code and documentation, and accessible stakeholder products. 

  • Ability to facilitate participatory interpretation and CQI processes with caregivers, youth, providers, agency leaders, and policymakers.

Work Location 

This is a remote position. The successful candidate must reside in Tennessee and be able to work remotely from a Tennessee residence. Occasional travel throughout Tennessee may be required.

Compensation and Benefits 

• UT market range: MR11
• Anticipated hiring range: $80,000.00
• Find more information on the UT Market Range structure here
• Find more information on UT Benefits here

Application Instructions 

Applications will be reviewed as received and interviews can begin as soon as 7 days from posting.

Interested individuals should provide the following:

• Resume
• Cover Letter

About The College/Department/Division 

 The Social Work Office of Research and Public Service (SWORPS) is an applied research center of the University of Tennessee College of Social Work (CSW).  We are a dynamic and close-knit interdisciplinary team of individuals with a shared vision of thriving communities with equitable outcomes for all. We do that by building partnerships that leverage research, technology, and human connection to improve lives in Tennessee and beyond.

Over the past fifty years, SWORPS has worked with local nonprofits such as the United Way of Greater Knoxville, University of Tennessee departments and faculty researchers, state agencies such as the Tennessee Department of Human Services, and federal agencies like the Department of Housing and Urban Development. Our annual portfolio consists of 25-40 sponsored projects spanning a wealth of different content areas, but one common thread -each project contributes one more building block for thriving communities with equitable outcomes for all.

Learn more about SWORPS and some of our programs and resources by visiting our webpage: https://www.sworps.tennessee.edu/