Agenda

Institute of Education Sciences Summer Research Training Institute: Single-Case Intervention Design and Analysis Agenda

Instructors: John Ferron, Ann Kaiser, Tom Kratochwill, Joel Levin, Wendy Machalicek, James Pustejovsky

Attendees: Early and mid-career researchers

Daily Schedule: 9:00 AM to 5:00 PM on Tuesday, June 18th and 8:00 AM to 5:00 PM thereafter, with breaks and 75 minutes for lunch; 8:00 AM to 1:00 PM on Saturday, June 22nd.

Group Activity: The Group Activity assignment is designed to provide attendees with an opportunity to gain experience in applying the concepts and strategies that are presented in the various instructional sessions. With the assistance of the instructors, the activity will mimic the process of developing a feasible and technically sound proposal to conduct a single-case intervention study. Each group will formulate an intervention strategy in a substantive area of its choosing, decide on the type of single-case design that is most appropriate, articulate the theory of action, and specify intervention and design options. As the sessions progress, the group will have an opportunity to apply the design, sampling, measurement, and data-analysis concepts that are discussed in the instructional sessions. Continuous feedback and resources will be made available to assist in the development of the study’s design and analysis. On the last day of the training institute, each group will present their final design and proposed analysis to the full assembly.

Consultation: A portion of each day will be allocated for the instructors to consult with Institute participants on the design and analysis of their ongoing and planned single-case intervention research projects.

Download the Agenda here

Day 1: Tuesday, June 18, 2019

8:00-9:00 AM – Registration

9:00-9:30 AM – Introductions and General Overview (Tom Kratochwill)

  1. Welcome and Introduction of Instructors and Participants
  2. Goals and Objectives of the Single-Case Design Institute
  3. Schedule and Logistics of the Meeting
  4. Group Activity Sessions and Individual Project Consultation Options
  5. Resources on Single-Case Design and Analysis

9:30-10:00 AM – Single-Case Intervention Research Opportunities at IES (Joan McLaughlin, National Center for Special Education Research)

10:00-10:30 AM – Logic and Foundations of Credible Intervention Research (Joel Levin)

Brief Review of the Principles of Scientifically Credible Intervention Research and Transition to Single-Case Intervention Research

10:30-10:45 AM – BREAK

10:45-11:45 AM – Tom Kratochwill

  1. Options for Single-Case Designs in Intervention Research
  2. Purposes and Fundamental Assumptions of Single-Case Intervention Research Methods
  3. Distinctions between Experimental and Clinical Case Study Designs

11:45-12:00 PM – Questions and Discussion

12:00-1:15 PM – LUNCH BREAK 

1:15 – 2:15 PM – Constructing Single-Case Designs Based on Research Questions and Logic Models (Wendy Machalicek) 

  1. The Importance of Theory of Change and Logic Models in Research
  2. Examples of Logic Models in Single-Case Research Design

2:15 – 3:30 PM – Appraisal Guidelines for Single-Case Designs (Tom Kratochwill)

  1. Rationale for Appraisal Guidelines
  2. Summary of Current Guidelines
  3. Applications of Single-Case Design Standards in Literature Reviews
    1. Example applications
    2. Advantages and limitations of the What Works Clearinghouse Standards
    3. Criteria for design credibility
    4. Criteria for evidence credibility
  4. Negative Results and Publication Bias
    1. Negative results
    2. Publication bias
  5. Replication Research
    1. The importance of replication research
    2. Direct replication
    3. Systematic replication
    4. Clinical replication

3:30-3:45 PM – BREAK

3:45-5:00 PM – Overview of the Group Activity (Wendy Machalicek and Joel Levin) and Initial Meetings (All Instructors)

5:00-5:15 PM – Group “Share and Tell” Session

5:15-5:45 PM – Individual Project Consultations (All Instructors)

5:45 PM – Social Hour with the Instructors

Day 2: Wednesday, June 19, 2019

8:00-8:30 AM – Individual Project Consultation (All Instructors)

8:30-8:45 AM – Recap of Day 1 Session and Overview of Day 2 Sessions (Wendy Machalick)

8:45-10:30 AM – Complex Single-Case Intervention Designs Continued and Overview of Designs used in Drug Research (Wendy Machalicek and Tom Kratochwill)

  1. Complex and Unique Single-Case Designs
    1. Changing Criterion Designs and Examples
    2. Multiple-Probe Designs and Examples
    3. Complex Phase-Change Designs and Examples
    4. Combined Designs and Examples
  2. Overview of Designs used in Drug Research
    1. Drug Design Options
    2. Placebo Controls
    3. Carryover Effects
    4. Single-and Double-Blind Assessments

10:30-10:45 AM – BREAK

10:45-11:45 AM – Randomized Single-Case Intervention Designs (Joel Levin)

  1. Rationale for randomization in single-case intervention research
  2. Randomization variations in single-case intervention designs
  3. Specific randomized single-case intervention designs

11:45-12:00 PM – Questions and Discussion

12:00-1:15 PM – LUNCH BREAK

1:15-3:30 PM – Visual Analysis of Single-Case Intervention Data (Wendy Machalicek and John Ferron)

  1. Overview of Visual Analysis of Single-Case Intervention Data (Wendy Machalicek)
    1. Research on visual analysis
    2. Advantages and limitations of visual analysis
    3. Training to improve the reliability and validity of visual analysis
  2. Visual Analysis of Single-Case Intervention Data (Wendy Machalicek)
    1. Parameters associated with visual analysis of single-case data displays
      1. The four steps of visual analysis
      2. The six features of visual analysis
      3. Data patterns in visual analysis
      4. Special considerations in visual analysis
  3. Issues and New Developments in Visual Analysis (John Ferron)
    1. Response-guided research designs
    2. Use of randomization in visual analysis
    3. Masked visual analysis and practice in their application

3:30-3:45 PM – BREAK

3:45-4:45 PM – Group Activity (All Instructors)

4:45-5:00 PM – Group “Share and Tell” Session

5:00-5:30 PM – Individual Project Consultation (All Instructors)

6:30 PM – Group Dinner at the Great Dane Downtown

Day 3: Thursday, June 20, 2019

8:00-8:30 AM – Individual Project Consultation (All Instructors)

8:30-8:45 AM – Recap of Day 2 and Overview of Day 3 Sessions (Wendy Machalicek)

8:45-9:45 AM – Using Single-case Designs to Build Programs of Scholarship (Ann Kaiser)

  1. Multiple, yet related, research questions
  2. Using results from one study to guide future studies
  3. Early intervention as an example of a content area that has benefited from programmatic scholarship.
  4. Combining group and single-case design within a study – Practical constraints

9:45-10:00 AM – BREAK

11:00-11:45 AM – Statistical Analysis and effect size measures for single case data series (James Pustejovsky and John Ferron)

  1. Organizing data for statistical analysis (James Pustejovsky)
  2. Non-Overlap Measures Derived from Visual Analysis (James Pustejovsky and John Ferron)
    1. Overview and history
    2. Selective, critical review
    3. Effect size calculators
    4. Recommendations
  3. Statistical modeling for single data series (John Ferron)
    1. Linear regression analysis
    2. Single-case adaptations
    3. Non-linear models
  4. Within-Case Effect-Size Measures (James Pustejovsky and John Ferron)
    1. Rationale
    2. Within-case standardized mean difference
    3. Percentage change and log-response ratio
  5. Meta-analysis of cases within single studies (James Pustejovsky)

11:45-12:00 PM – Questions and Discussion

12:00-1:15 PM – LUNCH BREAK

1:15-2:00 PM – Meta-Analysis Activity (John Ferron and James Pustejovsky)

2:00-2:45 PM – Inferential Statistical Analysis of Single-Case Intervention Data (Joel Levin and John Ferron)

  1. General Overview and History of Statistical Analysis of Single-Case Intervention Data (Joel Levin)
    1. The “whys” and “whats” or statistical analysis
    2. Early efforts, issues, and concerns
    3. The statistical-conclusion validity of single-case data: The elephant in the room

2:45-3:00 PM – BREAK

3:00-4:30 PM – Statistical Randomization Techniques for Single-Case Intervention Data (Joel Levin and John Ferron)

  1. Primer on nonparametric randomization tests
  2. Overview of single-case applications and randomization tests
  3. Conceptual and procedural issues
  4. Specific randomization test procedures
  5. Research applications
  6. The ExPRT computer package

4:30-5:30 PM – Group Activity (All Instructors)

5:30-6:00 PM – Individual Project Consultation (All Instructors)

6:00 PM – Informal Group Dinner at the Memorial Union Terrace (weather permitting) 

Day 4: Friday, June 21, 2019

8:00-8:30 AM – Individual Project Consultation (All Instructors)

8:30-8:45AM – Recap of Day 3 Presentations and Overview of Day 4 Sessions (Joel Levin)

8:45-10:00AM – Statistical Analysis and effect size measures for multiple data series (John Ferron, Joel Levin, and James Pustejovsky)

  1. Hierarchical Linear Modeling (John Ferron)
    1. Regression analysis and introduction to HLM
    2. Single-case adaptations of these procedures
    3. Conceptual and procedural issues
    4. Research applications
    5. Microcomputer software
  2. Between-Case Standardized Mean Differences (James Pustejovsky)
    1. Rationale
    2. Conceptual overview
    3. Limitations and future directions
    4. Software

Additional Comments and Examples (Joel Levin)

10:00-10:15 AM – BREAK

10:15-11:45 AM – Special Topics in Single-Case Intervention Research (Tom Kratochwill, Wendy Machalicek, and Joel Levin)

  1. Measurement issues
  2. Treatment integrity and intensity
  3. Cost-analysis
  4. Social validity in treatment research

11:45-12:00 PM – Questions and Discussion

12:00-1:15 PM – Lunch

1:15-2:30 PM – Special Topics in Single-Case Intervention Research Continued

  1. Protocol registry
  2. Selected ethical issues

2:30-2:45 PM – BREAK

2:45-3:45 PM – Small-Group Presentation Planning (All Instructors)

3:45-4:45 PM – Moving from Single-Case to Conventional Group Intervention Designs, Vice Versa, and Et cetera (Ann Kasiser)

  1. Emergent Design Standards
  2. Guidelines for Randomization, Blinding, Fidelity, and Statistical Analysis of Data
  3.  Additional Comments

4:45-5:00 PM – Questions and Discussion

5:00-5:30 PM – Individual Project Consultation (All Instructors)

Day 5: Saturday, June 22, 2019

8:00-8:30 AM – Individual Project Consultation (All Instructors)

8:30-9:00 AM – Recap of Day 4 Presentations and Overview of Day 5 Session (Tom Kratochwill and Joel Levin)

9:00-10:15 AM – Group Presentations

10:15-10:30 AM – BREAK

10:30-12:15 PM – Group Presentations and Faculty Feedback

12:15-12:45 PM Final Questions and Responses; Dissemination of Special Visual- Analysis Tools for Single-Case Intervention Research; Consultation with Instructors

12:45-1:00 PM – Participant Evaluation of the Training Institute

1:00 PM – DEPART TRAINING INSTITUTE