Close
About
FAQ
Home
Collections
Login
USC Login
Register
0
Selected
Invert selection
Deselect all
Deselect all
Click here to refresh results
Click here to refresh results
USC
/
Digital Library
/
University of Southern California Dissertations and Theses
/
Maximizing BCBA trainee development during supervision: knowledge, motivation, and organizational influences
(USC Thesis Other)
Maximizing BCBA trainee development during supervision: knowledge, motivation, and organizational influences
PDF
Download
Share
Open document
Flip pages
Contact Us
Contact Us
Copy asset link
Request this asset
Transcript (if available)
Content
Maximizing BCBA Trainee Development During Supervision: Knowledge, Motivation, and Organizational Influences by Carrie Ann Kahana Rossier School of Education University of Southern California A dissertation submitted to the faculty in partial fulfillment of the requirements for the degree of Doctor of Education May 2022 © Copyright by Carrie Ann Kahana 2022 All Rights Reserved The Committee for Your Full Name certifies the approval of this Dissertation Mary Andres Carey Regur Patricia Tobey, Committee Chair Rossier School of Education University of Southern California 2022 iv Abstract Clinical supervision is a formative experience for students working toward their board certification in behavior analysis. The purpose of the study was to learn about students’ supervision experiences, gain an understanding of current practices and preparation, and determine potential barriers to successful supervision. This study used the gap analysis framework to identify knowledge, motivation, and organizational factors that influence supervision experiences for aspiring board certified behavior analysts. A qualitative methods design included semi-structured interviews and document analysis with 10 students who were currently enrolled in behavior-analytic coursework at private universities in the northeast and accruing experience hours under a qualified BCBA supervisor. Based on the findings, this study offers recommendations to close validated knowledge, motivation, and organizational gaps using the New World Kirkpatrick Model to help BCBA trainees maximize their supervisory experiences. Keywords: clinical supervision, fieldwork, supervision, BCBA, trainee v Dedication To all those who keep going. You are stronger than you realize, and you can accomplish amazing things. vi Acknowledgements First, I would like to thank all the faculty at the Rossier School of Education whom I have had the pleasure to learn from. Specifically, I would like to thank Dr. Patricia Tobey, Dr. Mary Andres, and Dr. Carey Regur for sharing your time and knowledge while serving on my dissertation committee. Thank you, Dr. Marc Pritchard, for an impeccable Beatles reference but mostly for keeping me going. I would certainly not be here if it wasn’t for you. Thank you to my Cohort 15 colleagues for your encouragement, strength, and leadership. Thank you to my team at SEARCH Learning Group for your expertise, compassion, and commitment to our work and one another. Your presence made this possible. Thank you, Adam, for always providing me a space to emote and refuel. Your insight, humor, and kindness made all the difference. Thank you to my parents and family for their emotional support throughout this process, which at times felt insurmountable. Finally, I would like to thank my daughters, Zoe and Phoebe, for the honor of being their mom and for teaching me every single day. vii Table of Contents Abstract .......................................................................................................................................... iv Dedication ....................................................................................................................................... v Acknowledgements ........................................................................................................................ vi List of Tables .................................................................................................................................. x List of Figures ................................................................................................................................ xi List of Abbreviations .................................................................................................................... xii Maximizing BCBA Trainee Development During Supervision: Knowledge, Motivation, and Organizational Influences ............................................................................................................... 1 Context and Background of the Problem ............................................................................ 2 Organizational Goal ............................................................................................................ 3 Description of Stakeholder Groups ..................................................................................... 3 Stakeholder Group for the Study ........................................................................................ 4 Stakeholder Performance Goals .......................................................................................... 5 Purpose of the Project and Research Questions .................................................................. 6 Importance of the Study ...................................................................................................... 6 Overview of Theoretical Framework and Methodology .................................................... 7 Review of the Literature ................................................................................................................. 8 History of the Board Certified Behavior Analyst Credential .............................................. 8 History of Board Certified Behavior Analyst Supervision .............................................. 11 Effective Clinical Supervision from Related Fields ......................................................... 21 Gap Analysis Framework ................................................................................................. 27 Stakeholder Knowledge, Motivation, and Organizational Influences .............................. 28 viii Conceptual Framework ..................................................................................................... 40 Conclusion ........................................................................................................................ 43 Methodology ................................................................................................................................. 45 Research Setting ................................................................................................................ 46 Data Sources ..................................................................................................................... 46 Participants ........................................................................................................................ 48 Instrumentation ................................................................................................................. 49 Data Collection Procedures ............................................................................................... 50 Data Analysis .................................................................................................................... 51 Credibility and Trustworthiness ........................................................................................ 52 Findings ........................................................................................................................................ 53 Summary ........................................................................................................................... 73 Recommendations ......................................................................................................................... 73 Limitations and Delimitations ........................................................................................... 88 Recommendations for Future Research ............................................................................ 89 Conclusion ........................................................................................................................ 90 References ..................................................................................................................................... 91 Appendix A: Definitions ............................................................................................................. 103 Appendix B: The Researcher ...................................................................................................... 105 Appendix C: Instruments and Protocols ..................................................................................... 107 Pre-Screening Survey ...................................................................................................... 107 Interview Protocol ........................................................................................................... 111 Conclusion to the Interview ............................................................................................ 116 ix Document Request .......................................................................................................... 117 Participant Follow-Up ..................................................................................................... 118 Appendix D: Ethics ..................................................................................................................... 119 Appendix F: Consent Form ......................................................................................................... 121 Introduction ..................................................................................................................... 121 Key Information .............................................................................................................. 121 Detailed Information ....................................................................................................... 122 Statement of Consent ...................................................................................................... 125 Appendix H: Implementation and Evaluation Plan .................................................................... 127 Level 4: Results and Leading Indicators ......................................................................... 128 Level 3: Behavior ............................................................................................................ 130 Level 2: Learning ............................................................................................................ 134 Level 1: Reaction ............................................................................................................ 138 Evaluation Tools ............................................................................................................. 139 x List of Tables Table 1: Organizational Mission, Organizational Performance Goal, and Stakeholder Goal 5 Table 2: Knowledge Influences 32 Table 3: Motivation Influences 37 Table 4: Organizational Influences 40 Table 5: Data Sources 45 Table 6: Participant Demographic Information 54 Table 7: Supervision Document Analysis 69 Table 8: Summary of Knowledge Influences and Recommendations 75 Table 9: Summary of Motivation Influences and Recommendations 79 Table 10: Summary of Organizational Influences and Recommendations 85 Appendix E: Theoretical Framework Alignment Matrix 120 Appendix G: Coding Sheet with Themes 126 Table H1: Outcomes, Metrics, and Methods for External and Internal Outcomes 129 Table H2: Critical Behaviors, Metrics, Methods, and Timing for Evaluation 131 Table H3: Required Drivers to Support Critical Behaviors 132 Table G4: Evaluation of the Components of Learning for the Program 137 Table H5: Components to Measure Reactions to the Program 138 Table H6: Immediate Feedback Survey 139 Table H7: Delayed Feedback Survey 141 xi List of Figures Figure 1: Conceptual Framework 43 xii List of Abbreviations BACB Behavior Analyst Certification Board BCBA Board Certified Behavior Analyst IRB Institutional Review Board 1 Maximizing BCBA Trainee Development During Supervision: Knowledge, Motivation, and Organizational Influences Students pursuing the board certified behavior analyst (BCBA) credential do not undergo standardized supervision experiences while accruing the hours needed to fulfill Behavior Analyst Certification Board (BACB) requirements. The BACB regulates the BCBA credential, which requires a master’s degree in a related field, graduate-level behavior-analytic coursework through an accredited university, supervised clinical experience, and a competency exam (Garza et al., 2018). Even though the field of behavior analysis relies heavily on supervision to develop clinical skills in future behavior analysts, most BCBAs have not been provided with the training, mentoring, or support required to effectively prepare them to supervise others (Hartley et al., 2016; LeBlanc & Luiselli, 2016; Sellers, Valentino, & LeBlanc, 2016). In addition, many behavior analysts do not understand the critical nature of this responsibility and often replicate their own supervision experience even though it may have been poor; in turn, they are continuing the cycle of ineffective supervision behavior (Sellers, LeBlanc, & Valentino, 2016; Sellers, Valentino, & LeBlanc, 2016). The BACB Supervisor Training Curriculum Outline (BACB, 2018b) outlines supervision content but provides no guidance as to the specific activities or procedures to be used with trainees to ensure meaningful skill development (Garza et al., 2018; LeBlanc & Luiselli, 2016; Sellers, LeBlanc, & Valentino, 2016). There are few published resources to guide supervision practices and limited research on what procedures are most effective to deliver meaningful supervision (Sellers et al., 2019). This lack of specificity coupled with inadequate or absent supervisor training forces BCBAs serving as supervisors to select, design, and implement supervision procedures on their own, creating conflicting and inconsistent supervision 2 experiences for trainees. Even more, most recommendations focus on supervisor behavior, with minimal attention given to how to best prepare students for their BCBA supervision experience (Helvey et al., 2021). This problem is important to address because supervision experiences impact the quality of training for trainees, the behavior-analytic services they then provide to clients, the future aspirants who will be supervised by these BCBAs, and the future of the behavior analysis field (DiGennaro and Henley, 2015; LeBlanc & Luiselli, 2016; Sellers, Alai- Rosales, & MacDonald, 2016; Sellers, Valentino, & LeBlanc, 2016; Sellers et al., 2019; Turner et al., 2016). Context and Background of the Problem The BACB, a non-profit organization that manages the credentialing process for BCBAs, was established in 1998 to address the need for national certification for behavior analysts (Shook & Neisworth, 2005). Recognizing the critical nature of supervision, the BACB focused efforts on improving the supervision experience and continued to update requirements over the past two decades. These changes include additions in the number of experience hours required, guidelines regarding the process for documenting these hours, changes to the requirements to serve as a BCBA supervisor, ongoing continuing education requirements for those serving as supervisors, and the creation of a Supervisor Training Curriculum (BACB, 2018b) which outlines characteristics and skills needed to establish effective supervisory relationships (LeBlanc & Luiselli, 2016). LeBlanc and Luiselli (2016) highlighted that many changes have focused on the structure and content of supervision rather than practical application practice. This divergence leaves supervisors with the challenging task of designing their own procedures and effectively preparing students without any formal guidance or training, hence creating varied experiences for trainees. Researchers have focused recent efforts on identifying quality practices, 3 with a leading journal in the field, Behavior Analysis in Practice, dedicating a special section featuring seven papers in its December 2016 issue to effective practices. Organizational Goal The BACB (2018b) indicates supervision is designed to ensure the provision of quality services; foster the clinical, ethical, and professional development of future behavior analysts; and develop and maintain the trainee’s behavior-analytic skill set, all while serving as a model for effective supervision. The BACB requires BCBAs serving as supervisors to complete an initial 8-hour training via an approved continuing education provider, complete ongoing continuing education in supervision within each 2-year recertification cycle and use the BACB’s Supervisor Training Curriculum Outline (2.0) (2018b) to guide the supervision experience. Supervisors and trainees must document all accrued experience hours and tasks monthly, but there are currently no formal evaluation procedures to evaluate a trainee’s competency in any area or the effectiveness of the supervision received. Description of Stakeholder Groups Standardized BCBA supervision impacts several key stakeholder groups with influence on the field’s future success. First, BCBAs currently serving as supervisors may experience limited success without clear and well-defined processes designed to best support their trainees and their work. While these behavior analysts demonstrated the competency and experience required to become a BCBA, many have not had formal training on the specific skills needed to be an effective supervisor (Hartley et al., 2016; Sellers, Valentino, & LeBlanc, 2016). Second, university programs providing the coursework required to become a BCBA may not produce highly qualified clinicians if their students’ supervision experiences lack quality or consistency, impacting both the university and future employers. Universities must provide 4 coursework aligned with the current BCBA task list but have no obligation to facilitate or provide supervisors or settings whereby trainees can accrue experience hours. This lack of cohesion often leaves students to secure supervisors on their own, without direction or ongoing monitoring from faculty at the college or university program. Third, quality supervision and training impact clients who will receive services from these BCBAs in the future. Without proper training or clinical development, BCBAs cannot deliver quality and ethically-sound services, thereby influencing client progress and outcomes (Dixon et al., 2016; LeBlanc & Luiselli, 2016; Turner et al., 2016). Increased demand for BCBAs compounds this issue as there continue to be greater numbers of individuals in need of behavior-analytic services (Hartley et al., 2016; LeBlanc & Luiselli, 2016). Finally, and the focus of this study, lack of standardized supervision experiences affects students who are currently receiving BCBA supervision as part of their experience requirement. Supervision serves as the primary method of advancing a student’s skills from theory to practice and plays a critical role in an individual’s future development as a BCBA (Hartley et al., 2016; Sellers, Valentino, & LeBlanc, 2016; Turner et al., 2016). Students cannot become competent BCBAs without consistent and well-crafted supervision experiences, thereby highlighting their importance in both current and future success. Stakeholder Group for the Study The stakeholder group for this study is the students who are pursuing the BCBA credential and are currently receiving supervision from a qualified BCBA supervisor. The study focused on students enrolled in behavior-analytic coursework at private universities on the East Coast that deliver in-person instruction. This study targeted students who enrolled in behavior- analytic coursework and are accruing or who have completed accruing experience hours in 5 school, home, or center-based settings. Students possess a unique ability to directly influence the future field of behavior analysis. Aspiring BCBAs will become credentialed professionals and will be responsible for delivering high-quality services to clients, providing training and oversight to clinicians, and potentially providing supervision to future students pursuing certification as BCBAs. Failure to provide effective and streamlined experiences will impact not only these individuals, but future clients, supervisees, and the field of behavior analysis. Stakeholder Performance Goals The BACB utilizes its professional certification programs to ensure consumers receive quality services from credentialed practitioners. Table 1 highlights the BACB’s organizational performance goal, which supports this organizational mission. Table 1 Organizational Mission, Organizational Performance Goal, and Stakeholder Goal Organizational mission The BACB’s mission is to “protect consumers of behavior-analytic services worldwide by systematically establishing, promoting, and disseminating professional requirements” (BACB, 2020b, p.1). Organizational performance goal Students receiving BCBA supervision will undergo consistent and meaningful experiences which allow them to develop the clinical, professional, and ethical skills required. 6 Purpose of the Project and Research Questions Supervision continues to drive the development of the behavior analysis field and its clinicians. Effective supervision should develop both clinical (e.g., assessment, clinical programming and treatment, and ethical decision making) and professional skills (e.g., interpersonal skills, professional values, organizational and time management skills) for those pursuing certification (LeBlanc & Luiselli, 2016). While supervision requires a partnership between the supervisor and the trainee, much of the research and literature has focused on knowledge, motivational, or organizational gaps faced by the supervisor versus the trainee (Garza et al., 2018; Hajiaghamohseni et al., 2020; Hartley et al., 2016; Sellers, Valentino, & LeBlanc, 2016). Utilizing Clark and Estes’s (2008) gap analytical framework, the purpose of the project was to understand the experiences and potential barriers associated with supervision through the students’ eyes and how these ultimately influence and connect to the BACB’s goals for supervision and training. Two questions guided this study: 1. What are the knowledge, motivation, and organizational influences affecting student experiences while accruing supervision hours? 2. What are the recommendations regarding knowledge, motivation, and organizational resources to improve student experiences during supervision? Importance of the Study Current BCBA guidelines are vague and lack the standardization or specificity required to ensure supervisors are effective or that meaningful skill development occurs for BCBA trainees. The BACB created the BCBA credential, in part, to help identify qualified practitioners and protect consumers (Shook, 2005; Shook & Favell, 2008; Shook & Neisworth, 2005). Supervision experiences play a critical role in a candidate’s development of clinical, ethical, and 7 professional skill repertoires (Sellers, Alai-Rosales, & MacDonald, 2016). Inconsistent or poor experiences influence the establishment or mastery of these skills and have a lasting impact on a clinician’s future competence and, in turn, confidence in the BCBA credential. The newness of the certification juxtaposed with the rapid growth of the field and increasing demand for BCBAs magnifies the importance of quality supervision (Hajiaghamohseni et al., 2020; LeBlanc & Luiselli, 2016; Sellers, Valentino, & LeBlanc, 2016). The number of BCBAs continues to grow annually, further highlighting the need for high-quality supervision experiences for these trainees (BACB, n.d.-c). The totality of individuals impacted by these experiences extends well beyond the student and supervisor; rather, the reach includes future clients, future trainees, future employers, and the future of the behavior analysis field (Dixon et al., 2016). Quality and consistent supervision experiences for aspiring BCBAs are important to ensure the BACB is producing competent clinicians who can execute their professional, ethical, and clinical duties; failure to achieve this goal will impact the future development and success of the rapidly growing field, the professionals within it, and the clients they serve. Overview of Theoretical Framework and Methodology This study used Clark and Estes’s (2008) gap analysis model, which examines knowledge, motivational, and organizational gaps between current and desired performance. Clark and Estes emphasized identifying the human causes behind performance gaps that prevent organizations from achieving their goals. The gap analysis framework was used to identify gaps in student or trainee knowledge and motivation while simultaneously diagnosing organizational gaps in clinical placements and with supervisors. 8 To gain a deeper understanding of the learning, motivation, and organizational gaps influencing students’ BCBA supervision experiences, the study used a qualitative framework with a phenomenological research design (Creswell & Creswell, 2018). According to Merriam and Tisdell (2016), qualitative research requires that the researcher ask questions designed to collect and interpret data to understand participants’ experiences and perspectives. This study employed an in-depth review of relevant literature combined with semi-structured interviews and document analysis to identify solutions to increase consistency and improve experiences for aspiring BCBAs. Review of the Literature The following section provides a review of the literature on the BCBA credential, the role and process for supervising aspiring BCBAs, and barriers to effective BCBA supervision. The review then details effective clinical supervision practices from related fields, including their history, application, and purpose. The final section discusses the characteristics of clinical supervision before concluding with a summary of the impact of effective and ineffective practices. This review serves to help identify the knowledge, motivational, and organizational influences that facilitate successful BCBA supervision experiences. History of the Board Certified Behavior Analyst Credential The BACB manages the BCBA credential, including the processes for certification, credential maintenance, clinical requirements, and professional and ethical guidelines. This section details the creation and function of the BACB and provides an overview of the certification process. Finally, this section highlights the increasing demand for behavior-analytic services over the past 2 decades, which magnifies the importance of quality supervision experiences for students. 9 Creation of the Behavior Analyst Certification Board The BACB was established in 1998 to help address concerns in the field of behavior analysis surrounding the need to identify qualified practitioners and the ability to ensure quality service delivery to consumers (Shook & Favell, 2008). The mission of the BACB is to make and implement recommendations to promote best practices and protect those who receive behavior- analytic services (BACB, n.d.-b). The primary responsibility of the BACB is to manage the components of its behavior-analytic certification programs by establishing standards of practice, designating content to be covered during coursework and supervision, establishing ethical and professional guidelines, including procedures if these are violated, and managing exam content and administration. To execute this mission, the BACB offers various options for certification. Certification Process The BACB currently offers three distinct levels of certification accredited by the National Commission for Certifying Agencies: BCBA, board certified associate behavior analyst (BCaBA), and registered behavior technician (RBT; BACB, 2020b). The specific requirements for each credential vary, and the level of responsibility is differentiated within their respective scopes of practice. The BACB has continued to make updates to all areas of credentialing over the past 20 years. Most recently, the BACB updated the eligibility and examination requirements for those pursuing the BCBA certification, which impacted any candidate who applied on or after January 1, 2022 (BACB, 2020a). The current research study uses these updated certification requirements. The certification process for those seeking the BCBA credential includes components designed to address theoretical knowledge, applied knowledge, and demonstration of each via clinical experience and a competency exam (BACB, 2020B). The BCBA credentialing process 10 requires candidates to complete behavior-analytic coursework and hold a graduate degree in behavior analysis or a related field from a qualifying institution (BACB, 2020b). The BACB’s fifth edition task list (2017c) outlines content areas and hour requirements for graduate-level coursework. Individuals who already have a master’s degree can complete the coursework via a verified course sequence, and those who do not may satisfy both requirements by enrolling in an Association for Behavior Analysis International (ABAI) accredited graduate program that embeds coursework within a master’s program (BACB, 2020a). Individuals pursuing the BCBA certification must complete supervised applied behavior analysis fieldwork under the guidance of a qualified supervisor (BACB, 2020a). Candidates have the option of completing supervised or concentrated supervised fieldwork (BACB, 2020a). The latter allows individuals to complete the fieldwork requirement with fewer hours by increasing the amount and frequency of contact with their supervisor. Once candidates satisfy coursework and fieldwork requirements, they submit their documentation to the BACB for approval prior to sitting for a competency examination (BACB, 2020a). Once credentialed, BCBAs are required to complete 32 hours of continuing education every 2 years to maintain certification (BACB, 2020b). The credentialing process has evolved since the creation of the BCBA credential to maintain quality as well as keep up with demand for services. BCBA Demand The number of individuals holding the BCBA certification continued to increase annually since the creation of the BACB in 1998, thus making the need for quality and consistent supervision more significant. The behavior analysis field experienced significant growth since establishing the credential, with the overall number of BCBAs increasing from 22 in 1999 to 44,025 in 2021 (BACB, n.d.-c) and demand for those holding the BCBA credential increasing by 11 4,209% between 2010 and 2020 (BACB, 2021). The field’s rapid growth made the need for quality supervision more pressing, as there are greater numbers of students as well as more clients in need of behavior-analytic services. Individuals with the BCBA credential are qualified to use the science of behavior analysis to improve the lives of the clients they serve in a socially significant way (Baer et al., 1968). The scope of work for BCBAs includes goal selection, instructional procedure design and oversight, data analysis and program evaluation, and supervision and training of clinical staff (BACB, n.d.-d). The science of ABA has been documented to be effective with a wide range of populations across industries, including health and fitness, brain injury rehabilitation, behavioral sport psychology, treatment of substance abuse, and organizational behavior management (BACB, n.d.-a; Normand & Kohn, 2013). However, behavior analysts’ most common area of practice is with individuals with developmental disabilities, specifically autism (LeBlanc et al., 2012; Normand & Kohn, 2013; Shepley et al., 2017). Given the broad scope of work for behavior analysts, the need for quality supervision experiences for those pursuing the certification is critical to ensure meaningful and effective practices across clientele. History of Board Certified Behavior Analyst Supervision The BCBA supervision process requires clinicians to recognize its importance, understand the process, and successfully execute it in the field. This section discusses why supervision is important for aspiring BCBAs, the role it plays in their development, and the specific components of the process. This section examines current barriers to effective supervision and the steps the BACB has taken to address these challenges. Finally, this section summarizes the current research on effective practices and supervision efficacy 12 Importance of Supervision The field of behavior analysis relies on a supervision model to shape the repertoires of aspiring BCBAs, prepare them to deliver services to consumers effectively, and provide training and oversight to front-line staff (Sellers et al., 2019). Supervision, broadly defined, encompasses a wide range of activities, including oversight of a supervisee’s work, development of new skill sets, maintenance of previously learned skills, modeling data-based decision making, and setting an example for ethical behavior (LeBlanc & Luiselli, 2016). The supervisor is responsible for overseeing all areas of the trainee’s clinical, ethical, and professional skill development via ongoing contact over the course of an individual’s training experience and helping prepare candidates as they work toward the BCBA credential (LeBlanc & Luiselli, 2016; Sellers et al., 2019). The ability to provide effective and meaningful supervision is critical to the development of future behavior analysts and further growth of the behavior analysis field; however, there are few published resources to guide practices and limited research on what procedures are most effective to deliver meaningful supervision (Sellers et al., 2019; Sellers, Valentino, & LeBlanc, 2016; Turner et al., 2016). Despite the lack of direction on activities, the BACB does provide clear guidelines as to the formal supervision process. Supervision Process The BCBA certification process requires candidates to accrue supervised experience hours under the tutelage of a qualified BCBA supervisor. This section details the requirements both before and during the supervision process as well as considerations for selecting a supervisor, documenting experience hours, and the parameters of the process. 13 Eligibility and Onset The BACB outlines steps a trainee must complete before the supervision process can commence. The trainee needs to begin approved behavior-analytic coursework and identify a qualified BCBA supervisor prior to beginning formal supervision and accruement of experience hours. As of January 1, 2022, a qualified BCBA supervisor must be an active BCBA in good standing, have completed an 8-hour supervision training module, completed ongoing continuing education requirements, and have been certified as a BCBA for at least a year. If the individual has been certified for less than a year, they must meet with a consulting supervisor who possesses additional qualifications and experience for at least an hour per month. This 1-year certification requirement is an addition to the BACB requirements and was updated based on certification standards review and subsequent recommendations from subject matter experts (BACB, 2020a). Candidates receive guidance as to credentials the supervisor must possess but are offered limited direction on identifying a supervisor who can provide a high-quality and meaningful supervisory experience (Sellers et al., 2016; Turner et al., 2016). The BACB encourages trainees to gather information about potential supervisors, including length of certification, knowledge of supervisory requirements, efficacy with time management, and supervision style but offers no further direction (BACB, 2020a). Students can access a certificant registry on the BACB website, which lists all BCBAs nationwide and indicates whether they are willing to offer supervision. Students may email potential supervisor candidates directly or through employers, but they receive limited guidance on how to assess supervisory skill when selecting a supervisor (BACB, 2020a). 14 Students must enter a formal supervision contract with their supervisor at the onset of the supervisory relationship. This contract protects each party by clearly outlining the purpose of supervision, roles, and responsibilities for both the supervisor and trainee, training goals for the candidate, and circumstances whereby the relationship may be terminated. Supervised fieldwork can begin only after the trainee executes the contract with their supervisor (BACB, 2020a). Supervised Fieldwork Trainees working toward the BCBA credential must complete supervised fieldwork under a qualified supervisor to satisfy the experience requirements for certification (BACB, 2020a). The number of hours required ranges from 1,500 concentrated supervised fieldwork hours to 2,000 supervised fieldwork hours, depending on the number of contacts the trainee has with their supervisor each month. There are two types of experience hours: independent, whereby the supervisor is not present, and supervised, where the supervisor is present (BACB, 2020a). Trainees accumulate experience hours on a monthly basis, and they can accrue no less than 20 and no more than 130 hours per month. Trainees must participate in both restricted (e.g., with a client present, providing direct clinical intervention) and unrestricted activities (e.g., conducting behavior-analytic assessments, behavior-analytic program design, data analysis, training staff) with no more than 50% of the experience hours being restricted (BACB, 2020a). The BACB has specific requirements for documentation of all experience hours. Trainees must document all their independent and supervised hours in a unique documentation system to ensure they meet all requirements (BACB, 2020a). Further, monthly fieldwork forms detailing the nature of the candidate’s supervision must be completed and verified at the end of each supervisory period by both the trainee and the supervisor. Additionally, once the trainee completes all their experience hours, they create a final fieldwork verification form, which is 15 signed by both the trainee and supervisor (BACB, 2020b). Documentation systems for candidates capture the nature and duration of the supervision experience but do not address the quality or efficacy of training. Supervisors are directed to utilize the Supervisor Training Curriculum Outline (2.0) (BACB, 2018b) to guide their supervision practices. This document focuses on supervisor responsibilities and provides goals and objectives designed to facilitate an effective supervisory relationship and experience. Supervisors utilize the BCBA task list (fifth edition) to structure supervision sessions and help ensure their trainees develop the clinical, ethical, and professional skills required for competence. Effective supervision is critical to aspiring BCBAs’ development, but there are challenges to its execution. Barriers to Effective Supervision Without proper knowledge and training, the supervision process suffers, and students will not maximize their learning and development. Effective supervision requires understanding, careful planning, and skilled execution for both supervisor and supervisee. This section details current barriers to effective supervision for supervisors, trainees, and universities and their impact on student experiences and overall BCBA development. Supervisor Barriers Current BCBA supervision guidelines are vague and lack the standardization or specificity required to ensure supervisors are effective or that meaningful skill development occurs in supervisees. There are several challenges with the current BCBA supervision process for supervisors, including the fact that many BCBAs prefer to focus on clinical responsibilities over supervisory ones, have competing job responsibilities outside of supervision, and the realization that quality supervision requires dedicated time, training, and preparation (Hartley et 16 al., 2016). In addition to these barriers, there are few published resources to guide supervision practices and limited research on what procedures are most effective to deliver meaningful supervision (Sellers, LeBlanc, & Valentino, 2016). Recognizing the lack of formal guidelines, the BACB created a Supervision Task Force in 2011 to help identify supervision guidelines and areas to target (BACB, 2011b). The task force prompted changes to BACB candidates’ supervision and experience requirements, but the supervisor remained the sole decision-maker on supervision content and strategies. The BACB Supervisor Training Curriculum Outline guides current supervision practices (BACB, 2018b). While the BACB outlines supervision curriculum, there are no specific activities included to ensure effective teaching by the supervisor or acquisition by the supervisee (Garza et al., 2018; LeBlanc & Luiselli, 2016; Sellers, LeBlanc, & Valentino, 2016). Effective strategies are not dictated or outlined by the BACB, and individual supervisors are responsible for selecting, designing, and implementing procedures on their own. The lack of specificity coupled with BCBAs’ lack of formal supervisor training or experience creates a disconnect between the intended and actual outcome. Many behavior analysts have not received any guidance or instruction on how to provide effective supervision, do not understand the critical nature of this responsibility, and replicate their own supervision experience even though it may have been poor. In turn, they continue the cycle of ineffective supervision behavior (Sellers, LeBlanc, & Valentino, 2016; Sellers, Valentino, & LeBlanc, 2016). The current supervision process does not provide BCBAs with the training, mentoring, or support required to effectively prepare them to supervise others, and most BCBAs have not received any formal training on the specific skills needed to be an effective supervisor (Hartley et al., 2016; LeBlanc & Luiselli, 2016; Sellers, Valentino, & LeBlanc, 2016). 17 Quality BCBA supervision requires practices to be clearly defined and demonstrated to be effective through ongoing evaluation and evidence-based practices. Furthermore, effective supervisory experiences require training and preparation for both the supervisor and the trainee. Trainee Barriers There are currently limited resources dedicated to preparing students for their role in the BCBA supervisory relationship. While there are basic supports in place for supervisors, there are minimal training requirements to equip trainees with the knowledge needed to understand or successfully execute their role as trainees (BACB, 2017c). The BCBA 2022 Eligibility Requirements (BACB, 2020a) outline the supervision process for students, but the guidelines in the document focus primarily on the logistic elements of the supervision process (e.g., eligibility and required documentation) and do not offer clear or concrete guidance on how to maximize the supervisory experience from a clinical perspective. The BACB’s fifth edition task list (2017c) outlines eight basic objectives designed to prepare candidates to supervise others and requires 15 credit hours in supervision via coursework. The focus of this content is on how to be a supervisor versus how to be a supervisee. It is unclear if universities or supervisors effectively educate students on supervision goals and importance in their overall development as trainees, how the supervision process works and their role in it, the significance of these experiences in their professional growth, or the prioritization of skill maturation over time. Without a full understanding of the process, importance, and value of supervision, candidates may be merely fulfilling an experience requirement. It is important that trainees not focus solely on documentation and experience requirements. Rather, students need training on both the significance of effective supervision and best practices to ensure they receive it. 18 University Barriers Providing the behavior-analytic coursework designed to fulfill the requirements set forth by the BACB is the responsibility of ABAI-accredited programs (BACB, 2020a). To do so, university programs make changes to BCBA coursework based on recommendations from the BACB to ensure coursework aligns with the current BCBA task lists. While there is standardization to the content area requirements, there is a lack of uniformity in course delivery (e.g., on-campus, distance education) and in university provisions of supervisory experiences for students (Shepley et al., 2017). Universities do not need to provide or facilitate experience requirements, and trainees are often expected to secure supervisors on their own. Recognizing the many factors that influence supervision, recent efforts have focused on evaluating and understanding current supervision practices to help improve its effectiveness. Evolution of BCBA Supervision The BACB has made numerous updates to its certification standards over the past 20 years in alignment with its pledge to continually evaluate and evolve its systems and processes (BACB, 2012b). Since 1998, the BACB has adjusted many aspects of the certification process, including the creation of additional certifications, increased coursework requirements, updates to examination content, changes to qualifying degrees for eligibility, revised task lists, additions to continuing education requirements, and updates to the professional, ethical, and compliance code for behavior analysts. In addition to changes to these overarching areas, the BACB modified aspects of the supervision process to improve its quality and efficacy. The supervision process and requirements for supervisors and trainees have changed significantly since the creation of the BCBA credential in 1998. These changes include requirements regarding the increased number of experience and supervision hours, increased 19 contact with supervisors, and the stipulation that trainees can only begin supervision after coursework has begun (BACB, 2005, 2011a). Over the past 10 years, the BACB has continued to update supervision protocols. These changes include requiring 3 hours of continuing education on supervision for BCBAs during each recertification cycle, adjusting the number of supervision hours that can be accrued during each supervision period, and establishing parameters surrounding the start and end date of experience hours accruement. The BCBA also refined the processes for documenting supervision, including a formal supervision contract and fieldwork documentation forms to track ongoing supervision and feedback. An 8-hour supervision training module is required for supervisors, and a supervisor training curriculum provides clarification on the purpose and features of supervision as well as guidelines for delivering feedback and evaluating its effectiveness (BACB, 2012a, 2013, 2017a, 2017b). The BACB continues to update supervisory requirements for both supervisors and trainees, most recently adding two components that went into effect on January 1, 2022. Additional graduate coursework focuses on supervision content, policy outlining supervisors’ responsibilities to improve accountability, clarification on the nature of supervision, and the need for supervisors to observe and monitor trainees’ performance. Also, before serving as a supervisor to others, individuals need at least a years’ experience as a BCBA (BACB, 2020B). There have been few changes to the training or preparation provided to BCBA supervisors and trainees despite the many changes to supervision and experience requirements. While the BACB provides direction on the content, the strategies used and execution of the experience are left to the supervisor’s discretion and ability. Furthermore, the myriad changes to supervision requirements fall with the supervisor, leaving the student and the university without clear guidance, training, or accountability in the process. 20 Research on Effective BCBA Supervision There are currently few published resources to guide supervision practices and limited research on what procedures are most effective to deliver meaningful supervision. Effective supervision requires explicit instruction for the supervisor coupled with evidence-based resources to guide meaningful supervisory experiences (Sellers, Valentino, & LeBlanc, 2016). While research is sparse and there is no standardized approach to supervising trainees, recently, several supervision models have been offered based on the recognition of the current supervisory skill gap, including those focused on competency, apprenticeship, and group supervision (LeBlanc & Luiselli, 2016; Turner et al., 2016; Valentino et al., 2016). There is limited research on the efficacy of these suggested supervision models. There are commonalities among them, including establishing a structured supervisor-supervisee relationship; entering into a formal contract outlining roles, responsibilities, and expectations; clearly defining protocols and timelines for assessing supervisor and supervisee performance; and procedures to terminate the supervisor-supervisee relationship with plans to continue a professional relationship post-supervision (Garza et al., 2018; Hartley et al., 2016; Sellers, Valentino, & Leblanc, 2016; Turner et al., 2016). While the clinical application of behavior analysis is grounded in data-based decision making, evidence-based strategies for effective trainee supervision continue to remain largely absent. Garza et al. (2018) cited that while behavior analysts are fluent in using behavior change procedures with clients, they do not frequently apply these same tools with supervisees. The current Supervisor Training Curriculum Outline (BACB, 2018b) recommends using behavioral skills training as it is well-researched to be an effective technique for staff training (Parsons & Rollyson, 2012). Behavioral skills training provides a framework for training whereby a trainee 21 receives an explanation of a target skill vocally or in writing, the supervisor models the skill, and the trainee has opportunities for practice, followed by positive or corrective feedback (Turner et al., 2016). BCBAs could use the same strategies they use with clients with supervisees and supervisors to improve effective supervision behavior, therefore increasing their effectiveness and improving training outcomes (Garza et al., 2018; Parsons & Rollyson, 2012; Turner et al., 2016). Effective Clinical Supervision from Related Fields There is currently minimal data supporting the effectiveness of supervision practices in BCBA development (Turner et al., 2016). The current dearth of research in BCBA supervision practices and supervisee outcomes may be addressed by looking to other helping fields that also use supervised experiences to develop proficiency in aspiring professionals. Related fields such as medicine, social work, and psychology could serve as resources and provide additional knowledge about effective, evidence-based supervision practices (Sellers et al., 2019; Turner et al., 2016). The following sections detail the history of clinical supervision, the purpose of supervision, characteristics of effective supervision, and outcomes of both effective and ineffective supervision. History and Application Mental health professions have utilized clinical supervision as a key component of their training processes since the 19th century (Goodyear & Bernard, 1998). Clinical supervision plays a crucial role in the fields of psychoanalysis, family therapy, psychiatry, nursing, counseling, clinical psychology, medication education, and social work (Barnett, 2014; Ellis et al., 2015; Goodyear & Bernard, 1998; Jones, 2006; Kilminster & Jolly, 2000; O’Donovan et al., 2011; Sewell, 2018). Supervision promotes aspiring clinicians’ professional development and fosters 22 knowledge, communication skills development, and the technical application of skills learned via coursework and classroom training (Barnett, 2014; Tugendrajch et al., 2021). Many helping professions use both supervision and training to facilitate growth and professional development for trainees. Training differs from supervision by its focus and presentation. Goodyear and Bernard (1998) highlight that training typically occurs via classroom-based instruction without the client present and is more limited in scope, while supervision occurs when the supervisor observes the trainee delivering services to the client. This is an important differentiation as the successful transition from theory to practice is imperative for trainees to become effective practitioners and make the shift from student to professional. Purpose of Supervision Clinical supervision plays an integral role in the development of clinical, professional, and ethical decision-making abilities for aspiring professionals in the mental health fields. Supervision is typically provided by a senior-level professional to a junior-level supervisee and provides experiences that teach the supervisee to engage in professional and ethical practices (Falender et al., 2014). The supervisor’s responsibilities include evaluating the progress of the trainee, ensuring quality service delivery, and helping the trainee establish new skills (Barnett, 2014; Goodyear & Bernard, 1998; Jones, 2006; O’Donovan et al., 2011). From Student to Professional Effective clinicians must be able to integrate knowledge acquired via didactic instruction into their daily practice with clients. Supervision allows trainees to take the theoretical knowledge acquired via classroom-based instruction and apply these learned skills in the field (Pillay & Johnston, 2011). To establish clinical competence and close the gap between theory and practice, supervisors provide trainees with practice applying skills while working with 23 clients (Goodyear & Bernard, 1998). Supervision is the process whereby students become professionals and integrate the skills they have learned into their practical application (Gome et al., 2008). Supervisees should be provided with a wide range of experiences, balanced across content areas so that they can practice applying skills learned within coursework to clinical settings (Cole et al., 1981; Pillay & Johnston, 2011). Further, supervisors serve as models, mentors, evaluators, and teachers as supervisees transition into their respective professions and create the next generation of clinicians (Barnett, 2014). The focus on skill development and clinical competence throughout the supervision process is further emphasized by the supervisor’s responsibility to ensure clients are receiving quality services. Protecting Clients and the Future of the Field Clinical supervisors are responsible for protecting client welfare and ensuring their best interests are being met throughout the process. Supervision requires evaluation whereby supervisors assess supervisees on their readiness to work in a professional capacity (Goodyear & Bernard, 1998). Goodyear and Bernard further emphasize the important role supervision serves in considering a supervisee’s progress over time to determine if they are ready to move into the role of credentialed professional. The supervisor is not there to simply oversee training and development; rather, they must evaluate their supervisee’s growth to determine readiness for credentialing to help ensure future client welfare (Falender et al., 2014). The magnitude of the experience requires both supervisees and supervisors to engage in careful planning and to consider their individual roles in the process. Characteristics of Effective Supervision Supervision is a critical aspect of clinical training in many mental health professions, so it is important to identify factors that influence its efficacy. The supervisory alliance, supervisor 24 competence, supervisee experience level, and evaluation and feedback procedures all impact outcome and warrant consideration when preparing for a supervision experience. Supervisory Alliance The clinical supervision process is designed to transform classroom knowledge into clinical proficiency through an ongoing relationship between supervisor and supervisee. The importance of the supervisory relationship cannot be understated, and Ellis (1991) noted that supervisees identified it as the most important factor in supervision. A strong supervisory alliance is grounded in the recognition that the supervision process is designed to benefit both the supervisee and the clients they serve (Jones, 2006; Wickham, 2012). Effective supervisors make it clear to supervisees that their intention is to make them better and foster their development by delivering direct and clear feedback; providing guidance in clinical, ethical, and professional skill development; providing opportunities for collaboration and problem-solving; and serving as a role model (Goodyear & Bernard, 1998; Kilminster & Jolly, 2000; O’Donovan et al., 2011). To ensure supervisees feel comfortable reflecting on their own growth and challenges, supervisors must create a safe environment my modeling vulnerability and sharing their own strengths and weaknesses (Anderman & Anderman, 2006; Barnett & Molzon, 2014; Bernard & Goodyear, 2004; Borders, 2014; Rønnestad & Skovholt, 2013; Tugendrajch et al., 2021). Transparency and open communication between the parties creates an environment whereby trainees are more willing to engage in supervisee disclosure, admit mistakes, ask questions, and feel free to experiment (Barnett & Molzon, 2014; Bernard & Goodyear, 2004). Supervisor Competence Supervision is an essential aspect of clinical training and is one of the more frequent tasks completed by mental health professionals (Goodyear & Bernard, 1998). A supervisee’s 25 supervisory experience heavily influences their transition from student to professional, making it necessary for supervisors to have both clinical competence in their area of practice as well as competence in the skills needed to supervise others effectively (Barnett & Molzon, 2014). Having served as an effective practitioner or having received supervision does not lead to the ability or readiness to supervise others, as the ability to transfer one’s skills and knowledge to another requires a separate skill set (Barnett, 2014; Goodyear & Bernard, 1998). Supervision is recognized as a distinct clinical competency, and supervisors should not only demonstrate an interest in the supervisory role but be committed to their role as supervisors by investing in their own development via formal, didactic training in clinical supervision and remaining up to date with current literature (APA, 2015; Barnett, 2014; O’Donovan et al., 2011). Experience Level Effective supervision requires differentiated instruction and support based on a supervisee’s experience and current level of development. To ensure it is both meaningful and individualized, supervisors should assess supervisees’ training needs prior to beginning the formal process, as quality supervision requires a clear understanding of a supervisee’s strengths, weaknesses, previous clinical experience, and knowledge base (Barnett & Molzon, 2014). Individuals at the beginning of their supervisory experience benefit from more structured environments whereby supervisors provide support and guidance frequently and fade it contingent on the trainee’s confidence and experience (Goodyear & Bernard, 1998). Effective supervisors review their supervisee’s progress, reflect on their unique needs, evaluate the success of their support thus far, recognize the need for planned and purposeful activities, and recognize that supervision is developmental (APA, 2015; Borders, 2014; Goodyear and Bernard, 1998). Supervisors must realize that their support, direction, and training will be adjusted based on their 26 supervisee’s development and proficiency in each skill area. Effective supervision requires ongoing individualization and instruction tailored to the trainee’s specific needs, which will change over time (Barnett, 2014; Goodyear & Bernard, 1998). Evaluation and Feedback Effective supervisors engage in ongoing evaluation and feedback delivery to maximize supervisee development. Assessment is an essential component of supervision and facilitates goal development and focused training efforts (APA, 2015; Borders, 2014; Goodyear & Bernard, 1998). Supervisors following best practices inform supervisees about the rationale for evaluation, evaluation procedures and criteria, and the format and timing for feedback and assessment information delivery (APA, 2015; Borders, 2014; Kangos et al., 2018). The process should be structured with frequent meetings and opportunities for clear and focused feedback so the trainee can maximize learning during the supervision experience (Barnett & Molzon, 2014; Goodyear & Bernard, 1998; O’Donovan et al., 2011). Further, supervisees should be encouraged to engage in self-evaluation to identify their own areas of strengths and weaknesses, as well as identify areas in which they would like additional training (APA, 2015; Kangos et al., 2018), Outcomes of Effective and Ineffective Supervision The quality of a supervisee’s clinical experience can create either positive or negative outcomes. Effective experiences are linked to better client outcomes, higher levels of overall well-being for supervisees, increased supervisee self-efficacy, greater levels of skill acquisition, more advanced professional development, more positive views of the supervision experience, greater self-awareness, and greater adherence to clinical protocols (Bernard & Goodyear, 2004; Creaner, 2014; Falender et al., 2014; Gome et al., 2008). While effective and sound clinical 27 supervision is the goal, there are many instances where a supervisory experience does not meet these standards. At best, ineffective supervision may not equip the clinician with the skill set to complete their professional responsibilities. At worst, the experience may be harmful. The effects of negative clinical supervision include trainee dissatisfaction with the training experience, frustration with the supervisor, fewer candidates pursuing training in the future, contemplation of career changes, and higher levels of supervisee anxiety which influence their ability to both learn new and demonstrate mastered skills (Cole et al., 1981; Creaner, 2014; Goodyear & Bernard, 1998; Ramos-Sánchez et al., 2002). Even more, the effects of harmful supervision may lead to “psychological, emotional, and/or physical harm or trauma to the supervisee” (Ellis, 1991, p. 402), including shame, loss of confidence, health problems, or a decision to leave the profession (O’Donovan et al., 2011). The fact that supervisees may replicate their own experiences demands quality training and a commitment to ongoing evaluation and skill development. Gap Analysis Framework Clark and Estes’s (2008) gap analysis framework guides this study. The framework helps organizations achieve goals by identifying and addressing gaps between intended performance and actual performance. The model stresses the importance of operationalizing both individual and organizational goals and then identifying performance gaps that influence overall goal achievement. Clark and Estes identify three main causes of performance gaps: an individual’s lack of skill or knowledge (K), issues related to motivation (M), and organizational factors that may inadvertently impede success (O). The knowledge, motivation, and organizational (KMO) influences impact performance and must be clearly identified to facilitate goal achievement. Knowledge influences include 28 factual knowledge, conceptual knowledge, procedural knowledge, and metacognitive knowledge (Krathwohl, 2002). Motivation requires an individual’s active choice to begin a task as well as persistence in the task (Clark & Estes, 2008; Mayer, 2011; Pintrich, 2003). Principles that affect motivation may include self-efficacy, expectancy value, and goal orientation (Rueda, 2011). Finally, organizational factors such as culture, company procedures or processes, and resource allocation influence goal achievement specifically as they intersect with knowledge and motivation (Clark & Estes, 2008). All three areas must be analyzed together to effectively identify and close gaps between intended outcomes and actual outcomes for all involved stakeholders. This study utilizes this framework to identify KMO influences impacting BCBA trainees’ supervision experiences. Stakeholder Knowledge, Motivation, and Organizational Influences The KMO factors influence supervisee experiences. Students require conceptual, procedural, and metacognitive knowledge to take full advantage of the experience and maximize their development as professional behavior analysts. Regarding motivation, students must demonstrate self-efficacy, value the supervision experience, and possess a mastery mindset to facilitate their own growth. In addition, student experiences are influenced by the supervisors they are training under and the employers who create a supportive learning environment for trainees. Knowledge Influences Clark and Estes (2008) emphasized the importance of evaluating knowledge, as it relates directly to organizational and stakeholder goal achievement. Individuals must have both knowledge and skill to successfully perform and achieve their targeted outcome. For BCBA 29 trainees to maximize the supervision experience, students must have knowledge about their role and about the process, its goals, and anticipated outcomes. Krathwohl (2002) indicates that factual knowledge is the most basic type of knowledge, which serves as a foundation for more advanced knowledge types. Factual knowledge in BCBA trainee supervision experiences includes information such as familiarity with the content of the BCBA Task List (fifth edition), procedures for documenting fieldwork, and requirements for BCBA exam eligibility. Basic factual knowledge becomes more complex and interconnected as individuals develop conceptual knowledge. Conceptual knowledge for students includes a full understanding of why they are being supervised and what they should gain from the process. Procedural knowledge refers to how to do something, which, in this study, is how to engage in effective supervision and how to engage in behavior aligned with successful experiences. Finally, Krathwohl (2002) highlighted metacognitive knowledge, which refers to an individual’s awareness of their own thinking, their approach to cognitive tasks, and self-awareness. Knowledge of how to reflect on the supervisory experience and components of the training process are examples of metacognitive knowledge that efficacious students will possess. Successful experiences require students to possess procedural, conceptual, and metacognitive knowledge. Specifically, students must clearly understand how the process works, their role as a trainee, the rationale for supervision, what should be achieved because of the experience, and how to seek out activities that align with their strengths and weaknesses. Conceptual Knowledge Conceptual knowledge allows an individual to integrate the discrete components of factual knowledge to create a deeper understanding of process and application (Krathwohl, 2002). This understanding of the individual components and how they connect within the larger 30 infrastructure helps learners to have a fuller understanding of how a subject area is organized and how its various sub-sections work together (Anderson & Krathwohl, 2001). Conceptual knowledge, or knowledge of the what, has been shown to impact supervisee performance (Goodyear & Bernard, 1998). This study focuses on BCBA trainees’ knowledge surrounding the rationale for supervision, the goals of the experience, and the anticipated outcomes. A student’s conceptual knowledge influences their performance as a supervisee. Oftentimes, supervision presents a new experience for students who are accustomed to learning via classroom-based instruction, many of whom have never worked in professional or clinical settings before (Pillay & Johnston, 2011). Knowledge of the what as it relates to BCBA supervision is important, as Goodyear and Bernard (1998) note that trainees who have higher degrees of conceptual knowledge and understand the goals of clinical supervision are more interested in receiving feedback and direct coaching and less concerned with evaluation, thereby increasing the likelihood of positive and meaningful professional development. Procedural Knowledge Procedural knowledge pertains to an individual’s knowledge of how to complete a task, including understanding the underlying principles, techniques, and methods required for execution (Krathwohl, 2002; Pintrich, 2002). Procedural knowledge allows individuals to apply declarative knowledge and complete the steps necessary to execute the process or task (Anderson & Krathwohl, 2001). Students need to know the procedures for effectively navigating the supervisory relationship, have knowledge of effective supervision practices, and have a clear understanding of their role as a trainee. Trainees who demonstrate procedural knowledge understand their role as supervisees, their specific responsibilities, and the expectations of their supervisor (Kangos et al., 2018). 31 Direct and structured training can help them understand their role and responsibilities and the significance of supervision. Kangos et al. (2018) indicated that supervisees who grasp the process and everyone’s role within it are better able to maximize their experience, identify poor or harmful experiences, and advocate for both their own and their client’s needs. Without clarity in these areas, they may be unable to extrapolate maximum benefit from the experience or effectively identify if they are receiving quality supervision (Kangos et al., 2018; Reichelt & Skjerve, 2000). In addition, trainees who are unclear regarding their role or the expectations of their supervisor may experience increased anxiety, which has been shown to lead to a lack of skill acquisition, unwillingness to highlight areas of need, defensiveness, and conflict within the supervisory relationship (Ellis et al., 2015; Goodyear & Bernard, 1998). Metacognitive Knowledge Metacognitive knowledge is knowledge of the cognitive process, attentiveness to one’s knowledge and thinking, and includes concepts such as self-reflection and self-awareness (Pintrich, 2002). Metacognition can facilitate an individual’s knowledge of self as a learner and about the task and strategies needed to complete the task. It allows individuals to cope with new situations and helps promote learning transfer across settings (Anderson & Krathwohl, 2001). Metacognition is important for skill development, as the more one thinks about and is aware of one’s thoughts and cognitive processes, the more one can implement strategies to help their learning (Krathwohl, 2002). Reflective practices allow supervisees to continue to develop their clinical skill set. Educational and training programs for mental health and helping professions emphasize the importance of self-reflection, defined by Schön (1987) as the ability to reflect on one’s learning (Curtis et al., 2016). The lifelong process of active self-reflection is essential to a professional’s 32 ongoing development and provides opportunities to actively think about their practice, actions, and motivation, allowing identification of areas in need of growth or further education (Bernard & Goodyear, 2004; Ehrich et al., 2004; Falender et al., 2014). Students’ abilities to think about and understand their experiences, their needs, and how to best facilitate their development promote active inquiry, vulnerability, and ultimately a greater understanding of themselves and their role in all processes (Bernard & Goodyear, 2004). Those who engage in active and continuous reflection can change their clinical practices and attitudes surrounding aspects of their work (Schön, 1987) and can apply reflection tools throughout their career to support ongoing development (Bernard & Goodyear, 2004; Rønnestad & Skovholt, 2013). Those who engage in self-refection are more likely to be receptive to feedback, have higher degrees of self-efficacy, and better identify areas of need throughout the supervision process (Curtis et al., 2016; Falender et al., 2014). Table 2 presents the knowledge influences for successful supervision experiences for BCBA trainees. Table 2 Knowledge Influences Assumed Knowledge Influence Knowledge Type Trainees need to know why they are being supervised and what they should expect to learn from the experience (Goodyear & Bernard, 1998; Pillay & Johnston, 2011). Conceptual Trainees need to know how the supervision process works and understand their role as supervisee (Ellis et al., 2015; Goodyear & Bernard, 1998; Kangos et al., 2018; Reichelt & Skjerve, 2000). Procedural Trainees need to engage in self-reflection to identify areas of competence and areas in need of additional training. (Bernard & Goodyear, 2004; Cole et al., 1981; Ehrich et al., 2004; Falender et al., 2014). Metacognitive 33 Motivational Influences Knowledge helps us know what and how to do things, yet motivation drives us to do them. Clark and Estes (2008) identified three indexes that influence motivation and impact successful goal achievement: active choice, persistence, and mental effort. Active choice reflects an individual’s decision to, or not to pursue a specific goal. Once an individual decides to pursue the goal, they must then persist despite distractions or competing demands. Finally, motivation requires an individual to invest mental effort in working toward the goal, which is heavily influenced by experience and confidence (Clark & Estes, 2008). This study focuses on the need for students undergoing the supervision required for BCBA certification to demonstrate self- efficacy, have value for the supervision process, and possess a mastery mindset. Self-Efficacy Theory Bandura (1997) stated that self-efficacy is an individual’s belief in their ability to perform at a desired level and influences how an individual thinks, feels, and behaves. Self-efficacy is a critical aspect of motivation as higher levels of self-efficacy lead to improved motivation (Rueda, 2011). Greater self-efficacy positively impacts goal achievement as individuals are more likely to approach tasks with a mastery versus performance mindset, sustain effort despite setbacks or challenges, and recover from failures faster (Bandura, 1997). Bandura noted that self-efficacy can be increased through mastery experiences whereby individuals experience success after persevering through challenging circumstances, by observing social models who are similar to themselves sustain effort and experience success, and by social persuasion wherein individuals hear from others that they have the ability to complete the task. BCBA supervisees must believe in their ability to learn applied skills and make the transition from student to professional. The development of self-efficacy is critical as it provides 34 individuals with the motivation needed to persist with the clinical development process despite challenges along the way (Tompkins, 2013). Those with a higher level of self-efficacy expect a greater benefit from their supervision experience and require less supervisor-imposed structure (Goodyear & Bernard, 1998). While self-efficacy may vary with a trainee’s level of overall development, believing in one’s ability is essential to maximize learning, prevent imposter syndrome, and allow individuals to bridge the gap between theory and practice (Bernard & Goodyear, 2004; Goodyear & Bernard, 1998; Tompkins, 2013). Conversely, supervisees with low self-efficacy may doubt their ability to acquire skills, experience increased anxiety during supervision, and be unwilling to ask questions or seek training on areas that may be unclear (Goodyear & Bernard, 1998; Tompkins, 2013). Students must believe they are able to transfer knowledge and develop new skills during the supervision process, which facilitates the development of its utility value. Expectancy Value Theory Expectancy value theory views motivation as a function of both an individual’s expectancy for success and the value they have for the task (Eccles & Wigfield, 2002). A person’s belief in their ability to complete the task, combined with their desire to do so, offers powerful indicators of both success and performance (Wigfield et al., 2017). There are four key components of task value: attainment value, intrinsic value, utility value, and cost belief (Wigfield & Eccles, 2000). Attainment value refers to how the task aligns with an individual’s perception of who they are, what they believe, and whom they want to be. Intrinsic value reflects the enjoyment an individual experiences or expects to experience when completing the task. Utility value is how well a task fits into an individual’s goals, and it is experienced when an individual can accomplish personal goals because of task completion. Finally, cost belief related 35 to how much an individual must sacrifice to complete a task. Though all four aspects of value are influential, BCBA trainees may be most motivated by appreciating the utility and attainment value of their supervision experience. Supervisees must appreciate the ongoing and evolving nature of the supervision process and recognize its influence on their development as professionals. Students need to understand that supervision helps them transition their skill set from student to professional (Gome et al., 2008; Goodyear & Bernard, 1998). It is important for trainees to recognize that supervision, accompanied by ongoing feedback, will help prepare them for their work as BCBAs by connecting their theoretical knowledge to clinical application in the field (Goodyear & Bernard, 1998; Pillay & Johnston, 2011). Students who recognize the profound impact supervision plays in their development as BCBAs will be more likely to appreciate the process and its significance to both their current and future success (Barnett, 2014). By valuing the supervision process, students are best able to maximize their experience and demonstrate increased persistence and mental effort. Goal Orientation Theory Goal orientation theory offers insight into what motivates individuals to engage in learning and helps explain how they approach new tasks (Elliot et al., 2017). Elliot et al. (2017) noted that there are two basic types of goal orientations: mastery and performance. Individuals with mastery goal orientations are focused on learning new skills, developing competence, and self-improvement. Those with mastery, or growth, mindsets approach learning as a process and evaluate their progress by comparing their current level of proficiency to their prior accomplishments. By contrast, individuals with performance, or fixed, mindsets are focused on demonstrating competence and outperforming others; they view errors as an indication of failure 36 (Elliot et al., 2017). Students with mastery mindsets are more likely to remain engaged in learning because they believe they can develop new skills and are more resilient when faced with challenges (Elliot et al., 2017). Supervision is an ongoing, interactive process that allows trainees to develop proficiency as behavior analysts. Frequent evaluation and direct feedback are key components of effective supervision; supervisees who welcome feedback recognize that the supervision process promotes their learning and development as they transition from student to clinician (Bernard & Goodyear, 1998). A mastery mindset facilitates a trainee’s growth as it allows the student to focus more on establishing fluency in the application of behavior analysis versus demonstrating mastery via performance or evaluation (Rueda, 2011). In addition, a mastery mindset can help students learn to advocate on their own behalf by disclosing areas in need of future training or additional practice (Miehls et al., 2013). Conversely, a performance mindset can lead a supervisee to focus only on their areas of strength, leaving missed opportunities to identify and address areas of weakness or need (Bernard & Goodyear, 1998). Further, supervisees may become anxious if they focus too heavily on a supervisor’s perception of their performance and their desire for positive evaluations (Bernard & Goodyear, 1998). Students need to embrace a growth mindset and recognize that the goal of the supervisory experience extends far beyond the accruement of hours; it is about learning, growing, and becoming a skilled practitioner. Table 3 presents the identified motivation influences that shape successful supervision experiences for BCBA trainees. 37 Table 3 Motivation Influences Assumed motivation influence Motivation type Students need to demonstrate confidence in their ability to learn applied skills and connect theory to practice (Bernard & Goodyear, 2004; Gome et al., 2008; Goodyear & Bernard, 1998; Tompkins, 2013). Self-efficacy Supervisees need to prioritize the supervision experience and understand the critical role it plays in their future skill and identify as a BCBA (Gome et al., 2008; Goodyear & Bernard, 1998; Pillay & Johnston, 2011). Expectancy value Students need to focus on their clinical skill development versus the completion of required experience hours. (Bernard & Goodyear, 2004; Elliot et al., 2017; Miehls et al., 2013; Rueda, 2011). Goal orientation Organizational Influences People must have knowledge and motivation, but organizational influences also play a critical role in an individual’s ability to achieve both personal and organizational goals (Clark & Estes, 2008). Organizational processes, resources, and materials can offer meaningful support to individuals, but gaps in these areas may pose barriers to progress. Further, Gallimore and Goldenberg (2001) stated that organizational influences are reflected in cultural models and cultural settings, and these must be recognized to fully understand how they are potentially adding to the gap between desired and actual outcome. Cultural Settings: Employer Support for Supervision Cultural settings are the visible and tangible products that are derived from cultural models (Rueda, 2011). While cultural models define an organization’s beliefs or values, one can 38 observe cultural settings in the actions and activities taking place within the organization. Settings may present themselves as routines, policies, and procedures present in the organization (Gallimore & Goldenberg, 2001). Cultural settings are more concrete than cultural models, though models directly influence settings. The day-to-day behaviors and activities of organizations are influenced by cultural models but observed through cultural settings (Rueda, 2011). Trainees need to have a clear understanding of not only their role and responsibilities but also of the procedures that will be used during their BCBA supervision process. It is important that the supervisee is aware of the structure of supervision, activities that will be completed as part of their supervision experience, and the processes for review and evaluation and that the organization or supervisor clearly conveys this information to the trainee (APA, 2015). Systems such as supervision frequency, expectations and roles during supervision, supervision strategies, process for feedback delivery, meeting schedule, and expectations for trainee disclosure must be clearly articulated at the beginning of the supervision experience (Barnett & Molzon, 2014; Falender et al., 2014). Organizational support must facilitate effective supervision, as the supervisor’s competing responsibilities (e.g., large client caseload, too many supervisees, and limited time to devote to supervision and planning) can create challenges for BCBAs who are providing supervision (Barnett & Molzon, 2014). Cultural Models: Strong Supervisory Alliance Cultural models embody organizational values and influence both thoughts and actions. According to Rueda (2011), cultural models are shared mental schema that shape an organization’s understanding of how the world works. Cultural models help dictate the norm; they affect what the organization values and believes and how it operates (Gallimore & 39 Goldenberg, 2001). Cultural models are often abstract and develop slowly over time, both from shared experiences as well as inherited belief systems (Gallimore & Goldenberg, 2001). While cultural models may be difficult to directly observe, they are powerful as they shape an organization’s practices, policies, and even the relationships individuals have with one another. Organizations with cultural models that value collaboration, experimentation, and honesty can have a positive impact on BCBA supervision experiences. Organizations need to support the relationship between the supervisor and supervisee and recognize its influence on the overall impact of clinical supervision. A strong alliance between supervisor and supervisee is vital to effective practices (Bernard & Goodyear, 2004). Relationships that are based on trust, respect, and transparency have been shown to lead to maximum supervisee growth and higher quality service delivery to clients (Bernard & Goodyear, 2004; Falender et al., 2014). Trainees and supervisors who actively collaborate on goals and tasks to promote their achievement create stronger bonds, leading to increased engagement, attentiveness, and receptiveness (Barnett & Molzon, 2014; Bernard & Goodyear, 2004; Falender et al., 2014). Strong alliances create a positive and safe learning environment that leads to higher levels of satisfaction from the supervisee, more effective use of evaluation practices, increased supervisee willingness to share ideas, and more meaningful trainee disclosure during supervision (Barnett & Molzon, 2014; Bernard & Goodyear, 2004; Falender et al., 2014). Table 3 shows the assumed organizational influences that impact successful supervision experiences for BCBA trainees. 40 Table 4 Organizational Influences Assumed organizational influence Organizational category Students need to explicitly understand the procedures that will be used during their supervision process (APA, 2015; Barnett & Molzon, 2014; Falender et al., 2014) Cultural setting The organization needs to prioritize a quality alliance, clinically and relationally, between a supervisee and their supervisor (Barnett & Molzon, 2014; Bernard & Goodyear, 2004; Falender et al., 2014) Cultural model Conceptual Framework A conceptual framework offers a written or visual explanation of the key concepts, variables, and factors within a study and how they interact and intersect (Maxwell, 2013). The conceptual framework offers a lens through which the problem of practice is framed for investigation and helps identify how and why key variables are interrelated (Creswell & Creswell, 2018; Merriam & Tisdell, 2016). This framework provides a road map that guides the study based on previous research, key concepts in the literature, and the researcher’s beliefs and orientations. An understanding of the phenomena of interest allows the researcher to generate appropriate research questions, select the proper participants, design valid data collection methods, and analyze and interpret data (Maxwell, 2013). Clark and Estes (2008) asserted that KMO processes are the critical connection between an organization’s goals and its ability to achieve them. While an organization must address these three distinct areas, the KMO influences are interrelated and influence one another. For effective BCBA supervision to occur, students must possess the necessary knowledge, which, in turn, provides the motivation required to persist and put forth mental effort to achieve success. Finally, 41 it is critical that clinical placements and supervisors recognize their role and facilitate cultures that prioritize supervision and the many processes and considerations within. Students must have conceptual, procedural, and metacognitive knowledge to engage in successful supervision. Trainees who clearly understand the purpose of supervision, what they should expect to learn from the process, how the process works, and their role in it are more receptive to coaching and feedback (Kangos et al., 2018). When students focus on skill development versus evaluation, they increase opportunities for growth and can make more meaningful progress (Goodyear & Bernard, 1998). In addition, effective supervisor practices require metacognitive knowledge; that is, students need to be able to self-reflect to identify areas where they are confident as well as those that require additional training (APA, 2015; Bernard & Goodyear, 2004; Borders, 2014; Falender et al., 2014). Once students understand the what and how of supervision, they need motivation to make the most of their experiences. Students need to be motivated to participate and actively engage in the supervision process. Supervisees must believe they are able to successfully transition their skills from theory to practice and simultaneously recognize the key role supervision plays in this progression. Students who value the supervision experience as a pivotal factor in the development as BCBAs and feel alignment between the task and their perception of self will be more motivated to take an active role as a supervisee, persist despite challenges, and put in the necessary work to achieve their goals (Clark & Estes, 2008). Knowledge and motivation work in tandem. However, valuable supervision experiences also require organizational support. Organizational processes, supports, materials, and resources must be aligned with knowledge and motivational influences to promote effective supervision experiences. The structure of the organization itself can either help or hinder goal achievement and directly 42 impacts performance (Clark & Estes, 2008). Students may have the required knowledge and be well-motivated, but successful supervision requires support via an organization’s culture and values. Clinical placements and supervisors need to provide supervisees with clear descriptions of how the supervision experience within their specific organization. Materials and resources dedicated to trainee supervision, including providing supervisors and supervisees time to collaborate, ensuring supervisors have a manageable caseload, and tangible supports during the beginning, middle, and end of the supervisory experience are essential for goal achievement. Figure 1 provides a graphic representation of the KMO influences and how they impact and influence one another. This study evaluates how these individual factors and the synergy between them affect students’ supervision experiences. 43 Figure 1 Conceptual Framework Conclusion This study sought to identify resources that can best support students as they undergo supervision experiences while working toward their BCBA credential. I conducted a thorough review of relevant literature to help identify KMO influences that impact trainee’s experiences during supervision. An examination of the history of the BCBA credential and supervision process highlighted several barriers to effective supervision, including the lack of research on effective supervision practices for BCBA trainees. Despite ongoing changes and updates to the supervision process, poor supervision experiences continue to impede student development and the future of the behavior analysis field. Clinical supervision practices from related fields offer 44 insight as to KMO factors that shape supervision experiences. Knowledge influences include conceptual, procedural, and metacognitive knowledge; that is, students need to understand the goal of supervision, how to be effective supervisees, and their role in directing their own development. Motivational influences include self-efficacy regarding their own ability to transition from student to clinician, utility value, and recognizing the critical role supervision plays in their development as a BCBA, and goal orientation whereby students recognize the process of meaningful supervision is more valuable than the simple accruement of experience hours. Finally, organizational influences include students’ understanding of their supervision experience, including both processes and procedures, as well as the prioritization of a strong alliance between supervisee and supervisor. The literature and highlighted influences provide context to understand the problem of practice and inform the research questions for this study. 45 Methodology The purpose of this study was to understand the KMO influences that impact students’ experiences while accruing the supervision hours required for BCBA certification. The study was designed as a qualitative approach using semi-structured interviews and document analysis. To best answer the research questions, I conducted one-on-one interviews with 10 students currently receiving supervision under qualified BCBA supervisors to allow for rich discussions with members of the stakeholder group of interest (Merriam & Tisdell, 2016). Following the interviews, I reviewed and analyzed relevant documentation to help clarify student experiences and provide additional information as themes emerged (Merriam & Tisdell, 2016). Table 5 Data Sources Research questions Interviews Document analysis What are the knowledge, motivation, and organizational influences affecting student experiences while accruing supervision hours? X X What are the recommendations regarding knowledge, motivation, and organizational resources to improve student experiences during supervision? X 46 Research Setting The study took place at three universities in the northeast United States that provide the approved course sequences to fulfill the coursework required for the BCBA credential. The first was a private, Catholic university that offers students in-person instruction, which I refer to as University A. University A is currently home to approximately 6,000 undergraduate students and 4,000 graduate students and employs almost 1,100 instructional employees. University A currently has 27 students enrolled in the behavior-analytic coursework. The second university, which I refer to as University B, is also a private, Catholic university that offers in-person instruction. University B has a total enrollment of approximately 2,300 students (including both undergraduate and graduate) and employs approximately 100 faculty members. University B currently has 43 students enrolled in the coursework required for BCBA certification. The third university, referred to as University C, is a private university offering in-person instruction that serves approximately 4,000 students (including both undergraduate and graduate). University C currently employs approximately 250 full-time faculty and has 66 students enrolled in behavior- analytic coursework I selected these settings as they allowed for access to students currently accruing the required experience hours and undergoing supervision by BCBA supervisors. The program’s students have undergone or are undergoing the supervision process and bring with them personal experiences. In effect, they are a source of meaningful information to help identify the KMO influences impacting their supervisory experiences. Data Sources Interviews were the primary method of data collection for this study. Merriam and Tisdell (2016) noted that interviewing is necessary when we seek to learn “how people interpret the 47 world around them” (p. 108). Semi-structured interviews with current students undergoing BCBA supervision or those who have recently completed their experience hours provided the information needed to understand the KMO influences that shape their experiences. Additionally, the research included document analysis to gather additional data to help answer the study’s research questions (Merriam & Tisdell, 2016). Semi-structured Interviews Semi-structured interviews consisting of 39 questions were used to identify KMO factors that influence students’ BCBA supervision experiences. Interviews allowed me to respond to individual differences in student experiences and delve deeper into topics connected to the study’s conceptual framework. Semi-structured interviews provided the opportunity to craft questions ahead of time yet ask unplanned questions designed to explore relevant and significant concepts as they emerge (Merriam & Tisdell, 2016). Open-ended questions allowed participants to share their experiences in a way that promoted rich understanding, leading to meaningful data collection and analysis. Document Analysis I reviewed documents and artifacts provided by the study participants. Document analysis provided an opportunity to gather data on the type of information that is shared with trainees as part of their supervision experience. Document review provided evidence and allowed for a greater understanding of what systems are used to support students’ understanding of the supervision process, structure, and components (Merriam & Tisdell, 2016). Documents were selected based on relevance to the study and were reviewed for authenticity and accuracy. 48 Participants I recruited participants via purposeful sampling to select a sample from which I could readily access answers to the research questions (Merriam & Tisdell, 2016). I contacted each university’s behavior-analytic program coordinators to discuss participation in the research project. I was offered the opportunity to attend one session of each university’s practicum classes to discuss the study with potential participants. I met with each university’s students via Zoom where I provided an overview of my background, my areas of interest, and my study. I scheduled these visits with the university contacts during their scheduled weekly class time I presented a brief overview of the research project to students during class. After the overview was provided, the professors sent out the electronic survey designed to gather basic demographic information (e.g., age, gender, race) and screen for eligibility and willingness to participate in the study. I provided each professor the link to the Qualtrics survey beforehand so he/she could send it out to the students. I was not provided with student names or email addresses to maintain their anonymity. The scripted overview and screening survey is provided in Appendix C. Afterwards, I reviewed the survey results to determine student eligibility and willingness. I followed up with all participants who expressed willingness to participate in the study, regardless of eligibility. For students who were willing to participate but did not meet the inclusion criteria, I expressed gratitude but let them know they have not been selected to participate. I contacted interested students who met the study’s inclusion criteria via email and set up interviews which were conducted via Zoom. 49 Inclusion Criteria Participants in this study were students enrolled in behavior-analytic coursework and were either currently accruing experience hours under a qualified BCBA supervisor or had completed their experience hours but were not yet credentialed as BCBAs. These students offered the opportunity for information-rich dialogues whereby their actual experiences, beliefs, and perspectives will inform the research. Exclusion Criteria Students who had not yet begun accruing supervision hours or who had accrued less than 100 supervision hours were not included in this study. Additionally, students who were employed by my organization were not eligible to participate due to the potential influence this external power dynamic could cause. Instrumentation I was the primary instrument for data collection in this study. I used semi-structured interviews and an interview protocol consisting of 39 questions designed to identify KMO influences affecting students’ supervisory experiences. I piloted the interview questions with three individuals who would have qualified for the study and refined and reordered the interview questions because of this trial (Merriam & Tisdell, 2016). I asked between three and five questions under each category, designed to gather information regarding KMO influences outlined in the study’s conceptual framework. The protocol included open-ended knowledge questions, experience and behavior questions, and opinion and values questions designed to help understand the assumed KMO influences (Merriam & Tisdell, 2016; Patton, 2002). Semi-structured interviews allowed for flexibility in question order, exploration of novel issues identified within the interview, and the freedom to 50 explore salient issues in greater depth (Merriam & Tisdell, 2016). The interview protocol can be found in Appendix C. I utilized document analysis to gather additional information about the study participants’ supervision experiences. Documents, which include a wide range of written, visual, and digital images, were selected based on their ability to answer the study’s research questions (Merriam & Tisdell, 2016). Specifically, I reviewed any relevant course syllabi, supervision contracts, site agreements, and supervision handbooks or guidelines provided to students by their supervisor or clinical placement. All documents were first redacted prior to sharing with me to ensure anonymity for the participants as well as their supervisor and clinical placement. Data Collection Procedures I prioritized participant privacy and confidentiality throughout the research process. All data were stored on a password-protected cloud-based server. I utilized my USC Google Drive, which offered both password protection and dual-identification security measures. Throughout the study, no personal information was used (e.g., name, email address) and all identifying information was coded. This coding allowed the research team to connect data to a participant’s personal information while ensuring privacy. For data storage, I created a folder for each participant on the server and assigned each student a pseudonym so individuals could not be identified. Additionally, university information was coded to ensure anonymity (e.g., University A, University B). I conducted all interviews via Zoom, with each lasting between 45 and 60 minutes. I obtained verbal consent at the onset of each interview. To do so, I shared a copy of the study information and consent forms in the chat and reviewed each document with the participant. I asked for and obtained verbal consent prior to beginning the recording. I recorded interviews 51 with transcription using the Zoom functionality to allow for accurate review and permanent product data collection. I took notes by hand during each interview to bolster trustworthiness and credibility and to ensure I did not miss any important points or meaningful statements (Merriam & Tisdell, 2016). At the end of the interview, I provided the participant with a link to their coded Google Drive folder in preparation for document sharing for my document analysis. I put this link into the Zoom chat and provided vocal direction regarding what documents to share and required procedures to ensure their anonymity and privacy (e.g., redacting all documents prior to uploading to the Google Drive folder). I followed up with all participants via email after each interview to send a formal request for relevant documentation related to their supervision for my document analysis. I listed the requested documents and security procedures to ensure confidentiality. Specifically, I directed participants to redact the documents and remove or black out all identifying information (e.g., name, email address, supervisor name, clinical placement) prior to uploading to the drive. All data collected throughout the research process was destroyed at the conclusion of the study. Data Analysis The primary tool for data analysis was coding responses gathered during both interviews and via document review. I analyzed interview transcripts and created codes based on emerging themes and their alignment with specific KMO influences (Merriam & Tisdell, 2016). I used thematic and descriptive coding when reviewing relevant documents with a strong focus on alignment with my conceptual framework. Findings were shared with my dissertation committee as well in my finalized and published dissertation. 52 Credibility and Trustworthiness Credibility and trustworthiness are essential considerations in the realm of qualitative research. Credibility refers to the alignment between the methods used and how accurately the findings are represented (Merriam & Tisdell, 2016). Trustworthiness pertains to the level of confidence the reader has in the data and methods used (Maxwell, 2013). Both facets must be examined to help demonstrate that a study’s findings are representative of its participants’ contributions and that there is truth in the study’s findings. I took the following steps to increase credibility and trustworthiness throughout the course of my research. To maximize credibility and trustworthiness, I used triangulation, member checking, adequate engagement in data collection, thick and rich descriptions of data, and reflexivity. Triangulation, collecting and comparing multiple data points from different sources (Merriam & Tisdell, 2016), was accomplished by collecting data from individuals with a range of different experiences and combining interviews with document analysis. I used member checking by confirming with interviewees that my understanding of their experience or perspective was accurate. Additional member checks were completed by sharing an initial analysis of their experiences with participants to confirm their thoughts were represented accurately (Creswell & Creswell, 2018; Merriam & Tisdell, 2016). I adequately engaged in data collection by interviewing enough students to reach saturation (Merriam & Tisdell, 2016). I conducted interviews until clear themes emerged and no new information surfaced in subsequent interviews. I summarized interview data in a rich, descriptive way to provide a clear understanding of participants’ perspectives and how they connect to the study’s findings (Merriam & Tisdell, 2016). Finally, I acknowledge that I was the primary instrument for this study. As the instrument, I collected and analyzed all the data but also brough inherent biases 53 (Creswell & Creswell, 2018; Merriam & Tisdell, 2016). I engaged in self-reflection of my biases surrounding BCBA trainee supervision experiences and identified and shared these to facilitate trustworthiness. Findings The purpose of this research was to identify the KMO influences that affect students’ experiences while accruing the supervision hours required for the BCBA credential. This study used Clark and Estes’s (2008) gap analysis framework to identify deficits in current practices and make recommendations to help address the BACB’s goal of providing students with consistent and meaningful experiences to ensure the development of the clinical, professional, and ethical skills required to serve as a BCBA. This project sought to answer one primary research question to help guide recommendations: What are the KMO influences affecting student experiences while accruing supervision hours? This section describes the participating stakeholders and details findings regarding KMO influences. Specific findings are organized by influence type and aligned with the study’s conceptual framework. Participating Stakeholders The stakeholders for this study were students currently accruing experience hours under a qualified BCBA supervisor. Participants were enrolled in the required BCBA course sequence at one of three universities in the northeast United States. The researcher conducted semi-structured interviews with the 10 participants via Zoom, with document analysis thereafter. Table 6 identifies the participants’ age, university attended, number of approved BCBA supervisors they accrue hours under, and the approximate number of total experience hours accrued at the time of interview. 54 Table 6 Participant Demographic Information ID Age Ethnicity University Number of supervisors Approximate number of hours accrued Mary 23 White B 2 1,200 John 28 White C 3 450 Scott 24 White A 2 650 Jennifer 23 Latino C 2 1,300 Beth 28 White C 2 300 Carol 28 Asian C 2 300 Danielle 24 Black C 1 550 Hannah 25 White C 3 1,000 Tracy 28 White C 37* 700 Amy 24 Black C 3 1,300 * Actual number of supervisors reported by the participant Knowledge Influences Students answered interview questions designed to assess their conceptual, procedural, and metacognitive knowledge related to their supervision experience. Findings confirmed the presence of gaps within each of the three knowledge types identified in this study’s conceptual framework. Knowledge gaps are discussed below and highlight the need for a greater understanding of the purpose of supervision, clarity on the administrative procedures related to supervision and documentation, and the importance of ongoing self-reflection. 55 Conceptual Knowledge This section discusses relevant conceptual knowledge influences on BCBA trainees, including their understanding of the purpose of supervision and student expectations for their supervision experience. Understanding the Purpose of Supervision. Students do not fully understand the purpose of supervision and are unable to describe its’ clinical, professional, and ethical scope. Seven students referenced BCBA skill development as a key purpose of supervision, but none specifically identified the direct role supervision plays in establishing their overall clinical, professional, and ethical skill repertoires. Mary stated the purpose of supervision is to “prepare us to be BCBAs,” while Scott felt supervision should “teach you how to be a BCBA.” None of the students were taught the purpose of supervision. Instead, they offered personal statements as to how they came to know its purpose. Jennifer shared, “I discovered it on my own,” while John offered, “I’m piecing it together. It sounds right in my head.” Half of the students correctly cited competency development as an important reason for their supervisory experience. John stated that the purpose of supervision is to “create a competent individual that can help people … and to make sure you’re doing what you’re supposed to be doing.” Mary stated the goal of supervision is to “[make] sure what you’re doing is accurate.” Several other participants highlighted the importance of completing tasks associated with the BCBA credential within their supervisory experience. Danielle offered that supervision was designed to “get a better understanding of, and [learn] how to do the certain things we have to do as BCBAs.” Findings indicate students do not understand that a primary reason for supervision is to help them transition from student to practitioner. Four students highlighted the role of 56 supervision in transferring content learned via coursework to clinical work in the field. Jennifer explained that “one of the ultimate goals [of supervision] is just, like, applying what we learn in school into the field.” Carol discussed the role supervision plays for students as they develop their clinical skill set and detailed that it allows trainees to take “what we’re learning at school [and] .... apply it in a clinical setting and make sure we’re applying it correctly and getting feedback when needed.” Tracy further emphasized the concept of transferring theory to practice within supervision by stating, “you can read a book from front to finish and think you understand...and you never really know how to do it.” Supervisee Expectations for the Supervisory Experience. Findings indicate students approach their supervisory experience expecting to learn the skills required to serve as an effective BCBA. Students believe that supervision experiences will help them gain critical knowledge that will allow them to serve as behavior analysts. Hannah shared her belief that the supervision experience will allow her to “learn the job” and “become a competent BCBA.” Mary echoed these expectations and stated her expectation to “gain knowledge” and “become a competent BCBA.” Trainees’ desires and expectations to participate in tasks associated with the BCBA credential, however, are disconnected from opportunities to do so. Limited opportunities to participate in or receive direct training on the daily responsibilities of a behavior analyst were cited by over half of the study’s participants. Mary stated, “The part I feel like I’m missing right now is doing true BCBA work.” This point is further addressed within the realm of cultural settings. Procedural Knowledge Data analysis identified three procedural knowledge influences impacting BCBA trainees. This section details findings related to student knowledge of effective supervisory 57 practices, trainee roles and responsibilities, and student confusion related to mandatory BCBA paperwork and documentation requirements. Knowledge of Effective Supervisory Procedures. Interviews confirmed that students can identify effective supervisory procedures and describe proficient supervisor behaviors. Students identified behavioral skills training as an evidence-based supervision and training procedure and a component of effective supervision. Participants described important supervisory behaviors such as collaboration, modeling, and feedback delivery. Scott stated, “I think that the feedback c-component is definitely probably the most important.” Students identified consistent communication, direct instruction, and objective goal selection based on student strengths and weaknesses as effective supervisory behaviors. Student Understanding of the Supervisee’s Roles and Responsibilities. Data suggest that supervisees understand their role within the supervisory relationship. Students were able to accurately identify areas trainees are responsible for in both the clinical and administrative domains. More than half of respondents identified question-asking and identifying areas in need of further training as key trainee responsibilities. Beth explained, “I think asking those questions, especially things that I’m not understanding in class helps. I think asking those questions helps them help me.” Several students referenced the role they play in their own development and the importance of asking for additional training or support where required. Jennifer shared, “I think it’s super important … to take initiative … in telling your supervisor what you’re confused with or what you don’t know.” Danielle offered a similar sentiment regarding the supervisee’s role in actively seeking training in areas of weakness and the importance of “coming up with questions I have for my supervisor and asking him … to help me … because there’s a lot in this field that I 58 don’t know and need to know.” In addition to asking questions, participants identified seeking, accepting, or implementing feedback as one of their responsibilities as trainees. Trainees recognize the importance of vulnerability and personal development within their own learning. Tracy discussed the importance of “being open,” while Carol discussed the value and significance of feedback, “I always, always, always ask my supervisor, ‘What can I do better?’ or, ‘What did, what did you not like that I did?’” Several students also cited learning as one of their responsibilities, with Scott detailing, “I kinda have to be a bit of a sponge, like, take it all in.” Along with self-awareness and ongoing skill acquisition, students acknowledged the importance of accurate record-keeping and documentation. Respondents highlighted professional skills such as punctuality, organization, and accurate record-keeping as key responsibilities. Document analysis confirms the importance of these responsibilities and reveals a heavy emphasis on procedural and administrative expectations such as “[complying] with the company handbook,” “[adhering] to high standards of professional behavior,” and [staying] up-to-date with the BACB fieldwork requirements.” 1 Despite the need for accurate documentation and procedural adherence, students do not clearly understand the requirements or the processes for fulfillment. Confusion Regarding Administrative Procedures. Students’ responses indicated significant gaps in procedural knowledge surrounding the administrative practices relating to their supervisory experiences. Eight students noted challenges regarding paperwork, unclear understanding of required supervisory components, and ongoing changes to each made by the BACB. Participants repeatedly cited confusion surrounding how to log their supervision hours 1 Information derived from organizational handbooks; not cited to protect anonymity. 59 and poor understanding of the different types of hours required (e.g., unrestricted hours versus restricted hours). Mary noted, “There’s a lot to take in that the BACB puts out there, the handbook and everything. There’s a lot to unpack with that, and it’s hard to understand it.” Jennifer highlighted specific areas she felt frustrated and confused: “Do I need to sign up for accounts? What do I use to keep track of my hours? How many hours should I have on average, direct and indirect, or now it’s restricted and unrestricted.” Poor understanding and few resources for clear direction demonstrate a procedural knowledge gap in need of additional support. Further, students are not proficient in BCBA supervision requirements and do not receive sufficient training or support. Findings indicate that trainees do not receive adequate instruction on the documentation and experience requirements mandated by the BACB. Seven students referenced feeling confused or unclear about verification requirements and definitions, specifically referencing the distinction between restricted and unrestricted hours and the tracking system used for documenting their experience hours. Amy shared her experience at her supervision onset: All we had was a contract, and they never explained what unrestricted is, what direct is, what you should be doing for unrestricted. It was kind of like, here … have fun, kind of figure it out on your own. Other students received some training at the onset but wanted more ongoing support to confirm accuracy and that specific requirements are being met. Danielle explained, So, once I entered the relationship with my supervisor, she gave me the rundown of “This is what you need. This [many] hours. This is the least [number, this is the most [number] of hours you get a month. You need, like, two client contacts. You need to do two groups.” I wish, for the first month or two, [that] she was, like, “Okay, like, what day are 60 you free this week? We’re gonna do a client observation together.” Something like that, so I knew, “Okay, each month I’m going to this. This is what it looks like.” You know, something along those lines. Most trainees expressed they were not able to understand or successfully adhere to BACB documentation requirements. Metacognitive Knowledge The section discusses findings related to trainee processes for self-reflection and trainees’ abilities to seek help when needed. Processes for Self-Reflection and Self-Evaluation. The data suggested students do not have formal processes for ongoing self-reflection. While no participants actively engaged in any prescribed reflection, several students did discuss strategies they use to evaluate their progress and ongoing areas of need. Four students referenced reviewing the BACB task list to identify areas of competence and weakness. Mary detailed, “[My supervisor and I] sit down, and we go over the task list and see what I need help with or what I feel like I could use more practice with.” Similarly, Amy shared her strategy: “I use the task list to kind of go through … just kind of self-evaluate where I am [and] where I need help.” Students emphasized their recognition that there are areas in which they need additional training but noted they do not have any formalized processes for reflection or self-evaluation. Scott recalled a time when he was working with a student and questioned, “Am I doing this right?” Despite lacking systemized processes for self- evaluation, findings indicate students can request assistance and support when needed. Seeking Help When Needed. Interviews confirmed students demonstrated understanding and confidence in procedures to recruit additional support when needed. All participants cited their BCBA supervisor as their primary source of support when seeking help. 61 John shared his strategy for seeking help, “If I’m not confident running [a program], I’ll go seek my supervisor and be like, ‘I need you to do this with me.’” Mary offered that if she was unclear about something or felt like she needed more support, she would “go to one of my supervisors … and I would say, ‘Oh, I’m looking to learn more about this. Do you have the resources to help me learn more about this?’ And see if we could set something up.” Students offered strategies and processes for seeking help and findings confirm sufficient knowledge in this area. Motivation Influences This study assessed assumed motivational influences through a series of questions designed to evaluate students’ self-efficacy, utility value for the supervision experience, and goal orientation as they navigate their supervisory experiences. Findings suggest that while students consistently value and recognize the importance of supervision, students lack self-efficacy and do not embrace the mastery mindset required for maximizing their personal, professional, clinical, and ethical skill development. 62 Self-Efficacy Students answered several questions designed to assess their self-efficacy as it relates to their ability to connect theory to practice. This section discusses findings that indicate trainees lack confidence in their ability to transition from student to practitioner successfully. Trainee Confidence. Data suggest trainees display low self-efficacy and experience self- doubt in their ability to perform the clinical and professional responsibilities of a BCBA successfully. Seven students indicated they lacked confidence in their clinical skills and their ability to demonstrate technical knowledge and professionalism. Several students referenced self-doubt and shared their concerns about their ability to demonstrate competence. Tracy said, “I think I second-guess myself a lot for sure. I’m like, ‘I don’t think I can do this.’ And I’m like, ‘I don’t’ know if I could ever do that.” Jennifer summarized her current confidence level by stating, “If I were to go into the field tomorrow, independently as a BCBA, I’d be screwed. I feel that my skills are not where they need to be in the sense of paperwork and technical knowledge.” Danielle compared herself to her BCBA supervisors and stated, “I feel like I’m smart, but I feel like I’m not professional, in a sense, like they are. I’m nowhere near how smart they are.’” It should be further noted that eight students referenced feeling confused and overwhelmed at the beginning of their supervisory experiences. Participants cited confusion regarding administrative and paperwork procedures, which is directly correlated to this study’s findings indicating a lack of procedural knowledge in this area. Mary shared her feelings as she began her supervisory experience: “I felt clueless. Absolutely, like, in the dark.” Beth remembers the start of her supervision: “Oh, my god. It was the most confusing month of my life.” Along with confidence, trainees were asked about their value for the supervisory experience and its role in their development. 63 Value Study findings highlight two factors relating to value, which are addressed in the following section: trainee’s value for the supervision experience and trainee perception on supervisor influence. Trainee Perceptions on the Value of Supervision. Interviews confirm students value the supervisory experience and recognize its role in their overall development. Responses indicated students prioritize clinical supervision as it is aligned with their interests and goals. Students identified three main areas where supervision is most valuable: ongoing support, learning from their supervisor’s experiences, and promoting their learning from theory to practice. Four students cited their relationship with their supervisor and the ongoing guidance provided as a useful component of their supervision experience. Participants referenced the importance of meeting individually with their supervisor and knowing their supervisor is accessible. Danielle discussed the importance of knowing “I can go to my supervisor for anything, and he will always be there with an answer and will guide me.” Several students referenced appreciating their supervisor’s effort to provide them with direct training and the value of learning from someone who has direct experience in the role they are training for. Mary said: Like the experiences my supervisors have went through. I’ve never been through some of the things they have done. So, all the stories they had of different clients they’ve worked with. It’s different from reading it in a book because it’s more real. It’s more concrete and something I can relate to more. In addition, six students discussed the value supervision offers in connecting coursework to its real-world application in clinical settings. Scott shared his perspective on the importance of 64 supervision, “Coursework … is learning about what behavior analysis is, but the supervision experience is when you actually see all of these principles unfolding in person.” Hannah offered additional insight, stating that supervisors “provide the support that’s not in class. They’re the only ones that are actively seeing you apply principles and kind of molding you into what you’re going to be as a BCBA.” Along with a trainee’s value for supervision, students were also questioned about the impact their supervisor plays in their development as practitioners. Supervisor Influence on Trainee Development. Students recognize the importance of quality supervision, doing well within their supervision experience, and the critical role their supervisors play in their development and the BCBAs they ultimately become. The data indicated students possess attainment value and prioritize a successful supervision experience. Nine participants referenced the impact of their supervisor on their current and future success in the field. Scott offered, “Your supervisor has a huge, uh instrumental and profound effect on your development and future as a BCBA.” Hannah referenced the priority she placed on identifying a quality supervisor stating, “I knew I had to reach out to find the best experience possible. I was going to find people that were intelligent and were gonna be good mentors for me.” Several students further identified the impact their supervisor has on their impending role as trainers and supervisors to future BCBA trainees. Carol offered, I guess [my supervisor] is gonna have a really big effect on me, honestly, because I’m working right under her, so how she works, and how she responds, and how she implements, I think I’m pretty much going to mimic that. In addition, Scott noted, It’s almost like a cycle. If a supervisor trains a supervisee well, and then that supervisee becomes a BCBA and eventually becomes a BCBA supervisor, then I’m passing on what 65 I learned in my experience from my supervisor to supervisees who will then become future supervisors. Data indicate that students value the supervision experience, prioritize their success, and recognize the significant role their supervisor plays in their development. In addition to these facets of attainment and utility value, trainees shared thoughts about their approach to learning and navigating potential challenges within the process. Goal Orientation Trainees were asked a series of questions designed to learn about their approach to learning and understand their mindset as trainees. Findings indicate that trainees face competing priorities and have difficulty focusing primarily on their own skill development due to the BACB’s emphasis on hours accruement. Competing Priorities. Trainee interviews indicate students have conflicting priorities for their supervisory experience hours. Students cited several goals for their supervisory experience, which align with a mastery mindset, including establishing clinical competence and developing confidence. Scott offered that his approach as a supervisee is “to learn as much as possible so that I’m effective, compassionate, [and] ethical when I take that next step.” Jennifer summed up her goal for supervision as “knowing what I’m doing,” while Danielle wants “to know as much as I can.” Several students acknowledged that growth is a process and progress will occur over time. Scott stated, “Learning is something that’s never gonna stop, even if you think, as a supervisee, that you know a lot about a certain area. That doesn’t mean you’re automatically an expert on it.” In like manner, Carol reflected on her evolution as a trainee from the beginning of her supervisory experience to the present: 66 In the beginning, I wanted to make sure I was smart. I was scared and discouraged that I was never going to understand. I think once I started really applying it, it really started to make sense, so my mind has changed. My goals have become broader. In contrast, students also indicate BACB supervision requirements cause them to focus on task completion over progress or acquisition. Six participants indicated they prioritized accruing and accurately documenting their experience hours, some of which can be challenging to compile due to limited experience opportunities. Most students cited challenges with unrestricted hours, with John explaining, “All my colleagues are up in arms about getting indirect hours, like that is what everyone is stressing over. Everyone’s more focused on, like, growing the hours more than the concepts.” Jennifer emphasized the importance of these hours by stating, “You need your hours. You can’t sit for your boards without your hours. So, if you don’t get those hours, you’re screwed.” Mary shared her frustration with the heavy focus on hours and accurate documentation: “The way we track hours, I feel like so much time has been spent on that … when we should really be looking at our experience, not wondering what does this count as?” Student responses indicate there is a misalignment between mastery (skill development) and performance (hours accruement) mindsets, with students placing a greater emphasis on the number of their experience hours instead of on advancing their clinical repertoire. Organizational Influences This study’s conceptual framework highlighted the important role organizational settings and models have within the trainee experience. Assumed organizational influences identified in the literature were assessed during interviews, specifically exploring trainee understanding of supervision procedures they will experience within their organization and the prioritization of the 67 supervisory alliance. Findings indicate that organizations support the supervisory alliance for students and highlight a high level of trainee satisfaction. Conversely, results indicate a gap in trainees’ understanding surrounding the specifics of their supervision experiences which is discussed in the section below. Cultural Settings The following section discusses findings related to cultural settings, specifically identifying trainees’ confusion about supervision processes and organizational barriers to trainee development. Trainee Understanding of Organizational Supervision Processes. Organizations and supervisors do not effectively prepare trainees for their impending supervisory experience. Seven students felt the supervision process was not explained to them, and they did not know what supervision sessions would look like, what their role was within supervision sessions, or what to expect from their supervision experience. John described his experience as the onset of his supervision: “I don’t really know if they have a good plan for what’s going on. The first meeting was just kind of a mess. They didn’t really give good direction. They didn’t really explain too much about the process.” Amy shared her feelings at the beginning of her supervision, stating, “They didn’t tell me anything, so I’m like, ‘I don’t even know what I’m doing here.’” Students discussed the process of entering the required supervision contract with their supervisor/organization but explained that the contract was the only explanation they received about supervision expectations. Scott explained, “They kind of gave me the contract, and it was a really long contract, and they said, ‘If you have any questions, bring it back tomorrow, and we’ll go through the questions.’ We didn’t go through it together.” Additionally, document analysis confirms the information trainees received focused primarily on adherence to BACB procedures 68 and the administrative requirements of approved supervision experiences. Documents shared by students provided little to no explanation of the structure of supervision sessions, behavioral expectations for the trainee during their supervision experience, processes and timing for feedback delivery, or any specific information regarding scheduling or dates of supervision sessions, confirming students do not receive detailed information about their supervision experiences. A summary of this document analysis can be found in Table 7. 69 Table 7 Supervision Document Analysis ID Document received Lists specific supervision activities that will be used and behavioral expectations for trainees Details supervision schedule (e.g., dates, times, scheduling information) Details structure of supervision session Details how and when feedback will be delivered Mary Supervision contract No No No No John Supervision contract No No No No Scott Supervision contract No No No No Jennifer Supervision contract No No No No Beth Supervision contract No No No No Carol Supervision contract PowerPoint training for new supervisees No No No No Danielle Supervision contract No No No No Hannah Supervision contract No No No No Tracy Supervision contract No No No No Amy Supervision contract No No No No Barriers to Trainee Development. Findings indicate organizational barriers prevent trainees from maximizing their supervision experiences. Two primary barriers were identified: difficulty accruing unrestricted experience hours and a supervisor’s competing responsibilities within the organization. Data suggest trainees do not receive adequate practice opportunities to 70 participate in unrestricted activities. Students’ experience hours consist of restricted and unrestricted activities, and the latter are most closely associated with the job responsibilities of a BCBA. Seven participants referenced challenges with accruing unrestricted hours and the lack of opportunity within organizations for trainees to participate in activities that fall in this category. Mary discussed her experience with her organization: “They offer unrestricted hours, but it’s very, very, very difficult to get them. That’s the part I’ve been missing like most people.” Jennifer shared her experience with unrestricted hours: My skills are not where they need to be [with] paperwork and technical language, which is one of the main parts of being a BCBA. I’m not given those opportunities as often as I am with working with the child. Students expressed concern about their readiness to work in the BCBA position without sufficient training in these critical areas. Beth said, “I could fill up a whole page of direct hours. I wanna get the experience from my supervisor with my indirect hours before I move on and do them myself.” Interviews indicate supervisor caseload and competing responsibilities negatively impact a trainee’s supervisory experience. Eight students referenced limited time with their supervisor as a challenge to meaningful supervision. Participants cited supervisor caseloads, other job responsibilities, and limited time for supervision as areas of concern. Mary referenced seeing her supervisor “once in a blue moon” and described inconsistent supervision touchpoints, stating, “I would go weeks without being supervised.” Students discussed wanting more time with their supervisors. Tracy shared, “It would be better if we had more time,” and Danielle voiced, “I think it would make it better if we did see each other more often.” Participants made ongoing references to supervisors being busy with other supervisees and large caseloads. Mary explained, 71 “[My supervisor] has other clients and other therapists that serve them. So she needs to…give equal time to everyone.” Findings confirm organizational gaps exist in trainee development. Cultural Models Trainees were asked questions to learn about their organizations’ prioritization of the supervisory alliance. The following section discusses findings that indicate trainees and organizations prioritize this relationship. Supervisory Alliance. Trainees value the supervisory alliance and prioritize their relationship with their supervisor. Student responses highlighted four main relationship characteristics that were most valued: rapport, guidance, collaboration, and knowledge. All students emphasized the importance of establishing rapport to allow for a meaningful relationship with their supervisor. Amy shared, “I feel like it’s super meaningful to even just start out with a great relationship.” Trainees recognize the importance of a meaningful connection with their supervisor. Students shared the importance of feeling relaxed during supervision and that this allowed them to ask questions they might not otherwise ask. Mary explained, “She makes me feel very comfortable. I feel like I can go to her and talk to her about things. I’m not scared when she comes and observes me.” Beth echoed a similar sentiment and shared, “It’s nice that I know [my supervisor] well. It’s nice to not be scared to ask questions.” Two students shared opposite experiences where they did have a strong relationship with their supervisor and described the impact on their supervision. Amy recalled, “I was as nervous as I get for exams, like sitting there, I was so shaky, sweaty. I was like, ‘I’m gonna mess up.’ You can hear it in my voice, I was like, ‘Don’t mess up!’” Besides a supervisory relationship grounded in trust, students 72 appreciate supervisors who help trainees navigate the supervision process and their own skill progression. Students value a supervisor’s willingness to provide guidance and offer support. All students referenced their supervisors’ eagerness to give them individualized support with extra attention paid to their supervisor’s willingness to devote additional time, provide individualized attention. They feel like their supervisor genuinely cares for them. John described his supervisor’s level of support: “They’ll sit me down for like half an hour and just talk me through [a question]. They’ve gone out of their way to help me learn different parts of the task list independently on their own.” Danielle described his supervisor, stating, “He’s involved, and he cares. He wants to help me out. He wants to be a good supervisor, and he goes above and beyond to help me.” Along with accessibility and interest, trainees appreciate supervisors who know them as individuals and demonstrate concern for their progress. Students prioritize collaboration in their supervisory alliance. The supervisory alliance is a partnership, and six students referenced the importance of feeling like their supervisor values their input and knows their strengths and weaknesses. Mary said, “My supervisor is really open to suggestions,” and “I feel like she really hears me out.” Trainees referenced the importance of being aligned with their supervisors and how it positively impacts their development and overall experience. Tracy described the support provided by her supervisor: There’s a lot of things that I struggle with and so many things that I came to [my supervisor] with, and she was like, “We’re going to do this together. We’re going to figure this out.” So, that made it so much easier. The supervisory alliance provides students with a strong foundation for effective supervision, but the availability of ongoing training opportunities can impact trainee experiences 73 Summary The researcher conducted 10 semi-structured interviews and analyzed data gathered via document analysis to evaluate the assumed KMO influences impacting BCBA trainee supervision experiences. Data analysis allowed the researcher to generate findings across the targeted areas. The study confirmed gaps in two of the assumed knowledge influences and one new influence through data analysis. Findings confirmed that students lack conceptual knowledge relating to the purpose of supervision and metacognitive knowledge related to self- reflection. One new procedural knowledge gap was identified, which indicated that trainees lack clarity in the administrative and procedural requirements for supervision that are mandated by the BACB. Along with knowledge, gaps in motivation were also confirmed. Data analysis validated several gaps in motivation, including (a) student self-efficacy regarding connecting theory to practice and (b) misaligned goal orientation causing students to focus on hours accruement versus skill development. Finally, one expected organizational influence was confirmed, verifying confirmed trainees do not have an adequate understanding of the supervision process. One additional organizational factor was identified and highlighted several organizational barriers that prevent effective BCBA trainee development. The next section discusses and synthesizes these findings to answer this study’s second research question: What are the recommendations regarding KMO resources to improve student experiences during supervision? Recommendations This section details recommendations to help BCBA trainees maximize their development while participating in their required supervisory experiences. These recommendations are aligned with the study’s conceptual framework and organized by KMO 74 influences. The findings for each assumed influence are discussed, including whether the gap was validated and there is a confirmed deficit. A detailed implementation and evaluation plan utilizing the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016) can be found in Appendix H. This section concludes with a discussion of the study’s limitations and suggestions for future research. Knowledge Recommendations This study used semi-structured interviews and document analysis to validate assumed knowledge gaps that affect BCBA trainees during their supervision process. Findings confirmed two assumed knowledge influences and identified one new influence. Results validated gaps in conceptual knowledge and metacognitive knowledge and revealed an unexpected gap in procedural knowledge. Clark and Estes (2008) offered four approaches designed to close knowledge gaps that prevent organizational goal achievement. Information-sharing is the simplest type of information and provides individuals with details of what they need to know to do their job. Job aids provide individuals with information about how to complete a task and often take shape in checklists or process diagrams. Training is used when individuals require not only information and job aids but also demonstration, guided practice, and feedback to develop the skills for success. Finally, the most complex form of support is education. Education offers individuals theoretical information and strategies so that they can apply skills in unexpected or novel situations. The recommendations are designed to close the identified gaps in students’ conceptual, procedural, and metacognitive knowledge. A table detailing validated and unvalidated knowledge influences, knowledge type, supporting research identified in the literature, and recommendations to address these gaps can be found in Table 8. 75 Table 8 Summary of Knowledge Influences and Recommendations Assumed knowledge influence Knowledge type Gap validated (Yes or no) Principle and citation Context-specific recommendation Trainees need to understand the purpose of supervision. Conceptual Yes Trainees who comprehend the goals of their supervisory experiences are more likely to maximize their learning and development (Goodyear & Bernard, 1998). Provide trainees with information about the purpose of supervision specifically focusing on the goal or clinical, professional, and ethical skill development. Trainees need to understand how the supervision process works and their role as trainee. Procedural No To develop new skills, individuals must learn the discrete skills, practice applying them, and have the knowledge when to do so (Schraw & McCrudden, 2006). Gap not validated Trainees need to know how to complete the require documentation for the supervision experience. Procedural Yes, emergent To develop new skills, individuals must learn the discrete skills, practice applying them, and have the knowledge when to do so (Schraw & McCrudden, 2006). Provide trainees with information about the components of, and process for completing the require BACB supervision paperwork. 76 Assumed knowledge influence Knowledge type Gap validated (Yes or no) Principle and citation Context-specific recommendation Provide job aids in the form of checklists and flowcharts to support acquisition. Conduct training sessions to demonstrate paperwork completion. Provide opportunities for guided practice and feedback delivery. Trainees need to engage in self- reflection to identify areas in need of further development. Metacognitive Yes Metacognition helps individuals have more awareness and control of their own learning and promotes both motivation and acquisition (Baker, 2006). Students who think about their experiences and needs can better understand themselves and the role they play in their own development (Bernard & Goodyear, 2004). Provide training sessions to teach students to self-reflect on their ongoing development. 77 Conceptual Knowledge Interviews confirm a gap in conceptual knowledge, whereby students do not fully understand the purpose of supervision. Findings indicate students have a basic understanding of the goals of supervision but do not specifically understand the scope of its clinical, professional, and ethical influence. Student knowledge about the purpose of supervision is critical, as without a clear understanding of the purpose of their clinical supervision experience, supervisees are unable to maximize their development. Conceptual knowledge refers to a deeper and more complex integration of knowledge within processes and applications (Krathwohl, 2002). Trainees with high degrees of conceptual knowledge related to clinical supervision fully understand its goals and are more likely to recruit feedback and positively impact their own development (Goodyear & Bernard, 1998). Trainees should be provided with education and an explanation of the goal of supervision and the areas addressed within their experience. Students should receive both direct instruction and participate in small group activities where they can self-identify target skills in clinical, professional, and ethical domains Procedural Knowledge The assumed procedural knowledge gap in trainee understanding of the supervision process was not validated. Procedural knowledge refers to an individual’s understanding of how to complete a task and requires understanding the processes and procedures associated with the task (Krathwohl, 2002; Pintrich, 2002). A new procedural knowledge gap did emerge, however, and revealed students lack procedural knowledge related to proficiently completing the BACB’s required paperwork and documentation for supervision experiences. Additional training and 78 support is required for students to understand and successfully adhere to BACB experience and certification requirements. Interviews confirmed students are unclear about the definitions for different experience types and the processes for documentation. This lack of procedural knowledge will be addressed through training provided to students. Students should first be provided with information about the different types of hours that must be documented, including definitions and examples. Students should receive samples of the required forms and supports required for hours tracking. Additionally, students should receive job aids in the form of flow charts, which include the type of hours, examples of each, and clear directions for how to categorize and where to document. Finally, monthly group supervision meetings provide opportunities for continued training and support via modeling, guided practices, and opportunities for ongoing feedback while logging their own hours. These recommendations are designed to help students master new skills by first acquiring, then practicing, then applying the skill (Clark & Estes, 2008; Schraw & McCrudden, 2006). Metacognitive Knowledge Interviews revealed that students lack metacognitive knowledge and do not engage in ongoing self-reflection or have processes for their own self-evaluation. Metacognitive knowledge refers to an individual’s knowledge of learning and the cognitive processes required for acquiring new skills (Pintrich, 2002). Active self-reflection allows trainees to identify areas in need of further development and is a critical component of a trainee’s ongoing personal and professional growth (Bernard & Goodyear, 2004; Ehrich et al., 2004; Falender et al., 2014). Metacognitive knowledge gaps will be remediated via training sessions where students learn to engage in self-reflection. Trainees will learn the importance of self-reflection and the characteristics of the process (Schraw & McCrudden, 2006). Training will focus on 79 metacognitive strategies, including identifying prior knowledge, recognizing one’s own strengths and weaknesses, and actively reflecting on tasks upon completion. Instructional strategies include direct instruction, modeling self-reflection by thinking out loud, and providing opportunities for practice and feedback (APA, 2015; Mayer, 2011). Further, job aids, including checklists and graphic organizers, can be individualized and support trainee engagement in self- reflection. Motivation Recommendations This study evaluated three types of motivation influences: self-efficacy, utility value, and goal orientation. Gaps were validated in self-efficacy and goal orientation, though findings did not confirm a motivational gap in utility. A table detailing validated and unvalidated motivation influences, motivation type, context-specific recommendations to close these gaps, and supporting literature can be found in Table 9. Table 9 Summary of Motivation Influences and Recommendations Assumed motivation influence Motivation type Gap validated (Yes or no) Principle and citation Context-specific recommendation Trainees need to demonstrate confidence in their ability to apply skills learned in the classroom and connect theory to practice. Self-efficacy Yes Self-efficacy helps individuals select and commit to relevant tasks and increases persistence and motivation to complete them (Bandura, 1997; Eccles & Wigfield, 2002; Pintrich, 2003). Design scaffolded practice opportunities where trainees can practice applying skills. Incorporate direct coaching and feedback to facilitate growth and self-efficacy. 80 Assumed motivation influence Motivation type Gap validated (Yes or no) Principle and citation Context-specific recommendation High levels of self- efficacy and the belief in one’s own abilities helps students maximize learning, decrease anxiety during supervision, and successfully bridge the gap between theory and practice (Bernard & Goodyear, 2004; Goodyear & Bernard, 1998; Tompkins, 2013). Supervisees need to prioritize the supervision experience and understand the critical role it plays in their future skill as a BCBA (Gome et al., 2008; Goodyear & Bernard, 1998; Pillay & Johnston, 2011). Expectancy value No Learning and motivation are enhanced when learners value the task (Eccles, 2006; Pintrich, 2003). Personal interest and incorporating choice and control can positively impact motivation (Eccles, 2006). Gap not validated Students need to focus on their clinical skill development versus the completion of their experience hours (Bernard & Goal orientation Yes A mastery mindset promotes motivation, resilience, and engagement with learning by focusing on individual Teach trainees about the characteristics of mastery and performance mindsets. Create a safe and comfortable 81 Assumed motivation influence Motivation type Gap validated (Yes or no) Principle and citation Context-specific recommendation Goodyear, 2004; Elliot et al., 2017; Miehls et al., 2013; Rueda, 2011). improvement (Elliot et al., 2017; Pintrich, 2002; Yough & Anderman, 2006). A mastery mindset promotes growth by allowing trainees to focus on progress and fluency while recruiting additional training or support, where needed (Bernard & Goodyear, 1998; Miehls et al., 2013; Rueda, 2011). Create a learning community where individual support other efforts to learn (Yough & Anderman, 2006). Supervisors should create an environment where it is safe to take risks and model learning from their own errors (Anderman & Anderman, 2006). environment where students are encouraged to take risks and can safely focus on self- improvement. Provide opportunities to share and reflect on failures and growth via monthly group supervision sessions. 82 Self-Efficacy Findings validated a gap in students’ self-efficacy and confirmed that trainees’ lack confidence in their ability to transfer skills learned via coursework into their clinical practice. Self-efficacy, an individual’s belief in their ability to accomplish a task, is an essential element of motivation that promotes both commitment to the task and persistence despite setbacks (Bandura, 1997; Eccles & Wigfield, 2002; Pintrich, 2003). High levels of self-efficacy and the belief in one’s abilities help students maximize learning, decrease anxiety during supervision, and successfully bridge the gap between theory and practice (Bernard & Goodyear, 1998; Goodyear & Bernard, 2004; Tompkins, 2013). Trainee self-efficacy will be addressed through modeling, feedback, and scaffolding. Trainees will observe credentialed BCBAs successfully applying theoretical skills within the clinical setting, as self-efficacy can be increased by watching peers perceived to be credible (Pajares, 2006). Further, observing competent models provides trainees with vicarious experiences, which are an additional source of self-efficacy (Bandura, 2000). Trainees will engage in scaffolded practice opportunities designed to connect theory to practice. Students will receive ongoing feedback on their skill application, focusing specifically on areas of improvement and celebration of incremental success. Expectancy Value Findings indicate students effectively value the supervision experience, both as it relates to their future role as BCBA and in alignment with their own identities. Utility value is increased when individuals understand the relevance of their learning, while attainment value encourages alignment between tasks completed and an individual’s perception of themselves and their 83 competence (Wigfield & Eccles, 2000; Pintrich, 2006).) No gaps were identified in either utility or attainment value, so no recommendations are made in these areas. Goal Orientation Interviews confirmed trainees need a greater emphasis on clinical skill development instead of hours accruement during their supervisory experience. Findings highlighted students place a heavy focus on their number of accumulated hours and fail to properly attend to their own progress as aspiring BCBAs. Students need to focus on their development and progress, utilizing a mastery mindset. Trainees who embrace a mastery mindset are more focused on learning, understanding, and improving, all of which creates increased motivation and persistence (Pintrich, 2003; Yough & Anderman, 2006). Trainees who commit to developing fluency in their behavior-analytic skill repertoires are more likely to receive feedback on areas of need and persist despite failures, as they recognize these promote opportunities for growth (Bernard & Goodyear; Miehls et al., 2011; Rueda, 2011). Students should receive direct training on the characteristics of performance and mastery mindsets and the importance and benefits of having a mastery mindset. This training should take place at the onset of the student’s supervision experience but be revisited throughout the course of the trainee’s development at monthly group supervision meetings. Supervisors should model vulnerability and be transparent with their own areas of weakness and failure. It is important for trainees to observe models of how to process mistakes or setbacks and use them to promote their own growth. Supervisors need to create an environment where trainees can safely take risks and have the freedom to fail, knowing these are steps in their own growth and progress (Anderman & Anderman, 2006; Barnett & Molzon, 2014). It is also important to acknowledge the influence evaluation has on a supervisee’s willingness to disclose and display vulnerability while learning. 84 The supervisory relationship is evaluative in nature, and it is critical that trainees recognize feedback, coaching, and evaluation procedures are intended to promote their skill development (Barnett & Molzon, 2014; Goodyear & Bernard, 1998). Organizational Recommendations This study evaluated two organizational influences: one cultural setting influence and one cultural model influence. Findings validated the assumed cultural settings influence and identified an emergent settings influence. Data analysis confirmed that students do not receive sufficient information regarding the supervision procedures used during their supervision experience. Additionally, findings revealed and confirmed that students do not consistently perform tasks aligned with the BCBA credential. A gap in cultural models was not validated, as interviews established that organizations adequately support the supervisory alliance. A table detailing validated or unvalidated organizational influences, the influence type, principles and citations, and recommendations to close the organizational gaps identified in this study can be found in Table 10. 85 Table 10 Summary of Organizational Influences and Recommendations Assumed organizational influence Organizat ional type Gap validated (Yes or no) Principle and citation Context-specific recommendation Organizations need to prioritize a quality supervisory alliance between supervisors and supervisees Cultural models No Espoused values and beliefs must be consistent with organizational messages, relationships, and concrete practices (Clark & Estes, 2008; Gallimore & Goldenberg, 2001; Schein, 2004). Gap not validated Organizations need to provide trainees with details of their supervision experience including the structure of their supervision experience and procedures that will be used Cultural setting Yes Transparent and consistent communication regarding processes and systems improves performance (Clark & Estes, 2008). Create systems for ongoing information- sharing regarding supervision experience with students. Content should include the structure and content of supervision sessions, activities that will occur, processes for evaluation and feedback delivery, and information about scheduling, frequency, and duration. 86 Assumed organizational influence Organizat ional type Gap validated (Yes or no) Principle and citation Context-specific recommendation Organizations need to provide trainees with consistent opportunities to engage in tasks aligned with BCBA responsibilities Cultural setting Yes, emergent Resource and process alignment improve organizational performance (Clark & Estes, 2008). Identify and implement mechanisms that support ongoing training opportunities for students. Cultural Models The findings did not validate a gap in the assumed cultural model influence. Cultural models represent an organization’s shared thinking about how the world works, which is demonstrated within its policies, values, and practices (Rueda, 2011). In the realm of BCBA trainee development, cultural models influence how organizations prioritize the relationship between supervisor and trainee. A supervisor’s level of care and concern is positively correlated with a strong supervisory alliance and helps create a positive learning environment. The foundation set by rapport and connection allowed students to become comfortable around their supervisor, allowing for increased trust and vulnerability. Interviews confirm that organizations prioritize the supervisory alliance and that trainees feel connected and supported by their supervisors, so there are no recommendations. Cultural Settings Cultural settings are the day-to-day activities and actions within organizations (Rueda, 2011). Cultural settings are derived from an organization’s beliefs or values and can be seen it its policies and procedures. This study evaluated one assumed cultural setting: trainees’ 87 understanding of their specific supervision experience. This influence was confirmed, and an additional setting was identified as a barrier: trainee participation in BCBA tasks. The following section details recommendations for both. Organizational gaps currently prevent students from maximizing their supervisory experience and their BCBA development opportunities. Findings indicate supervisees are not provided with sufficient details to fully understand their impending supervisory experience. Interviews and document analysis confirm that students do not receive the information they need to prepare mentally, academically, and physically for their supervision experience. Students’ supervision contracts provide general information about their supervision experience, including the definition of supervision, sample activities that may be included, paperwork requirements, and minimum frequencies for different types of supervision but lack any specificity related to individual trainees. Students need to receive clear guidance and information on the timing and structure of their supervision, including a specific supervision schedule detailing dates and times, the format and frequency for feedback and evaluation, and behavioral expectations for the trainee. Students need to be provided with training on how to prepare for supervision sessions, what these supervision sessions will look like, and what their responsibilities are within the context of each type of supervision activity. While trainees broadly understand their role as supervisees, specific guidance should be given so they can generalize this understanding to specific activities to maximize their development. Recommendations include training and job aids to promote trainee understanding and comfort with the supervision process. Further, structural changes may be needed to support the organizational goal of developing future BCBAs. Creating an additional position within the organization, Supervision Experience 88 Coordinator, would facilitate implementing this study’s recommendations to close gaps in KMO influences. Data analysis identified an unexpected gap in accessing BCBA task opportunities. Interviews confirmed trainees have limited access to unrestricted hours, which provides training the skills most closely aligned with a BCBA’s responsibilities. This barrier can be attributed to a lack of supervisory availability due to competing supervisor responsibilities (e.g., large client caseload; too many supervisees) and misalignment with a trainee’s job responsibilities. Nine interviewees were employed by the organization providing their supervision hours and working in direct support positions (restricted hours), which preclude opportunities to complete tasks required to develop the full behavior-analytic skill set (unrestricted hours). Clark and Estes (2008) reference the importance of aligning structures and processes with goals. Therefore, recommendations to close this organizational gap include shifting the responsibility for the provision of indirect hours solely from the organization to increased recruitment for these opportunities by the trainee. Students should be taught to directly request opportunities to engage in BCBA-related tasks rather than waiting for the organization or supervisor to provide them. While organizations may also want to review competing supervisor responsibilities that inhibit BCBA task opportunities for trainees, the focus of this study was on factors over which trainees have direct influence. Limitations and Delimitations Every study has both limitations and delimitations because of its conceptual framework, the constructs selected, and the methods of investigation. Limitations are potential weaknesses to a study that are out of the researcher’s scope of control (Merriam & Tisdell, 2016). It is important to identify limitations as they may impact both the methodology and a researcher’s 89 ability to draw accurate conclusions. The limitations of this study included the participants’ willingness to answer questions openly and honestly, the small sample, variability in students’ levels of supervisory experience, variability in the number of supervisors under whom students are training, my potential unconscious bias as a BCBA myself, and the fact that some students received remote or atypical supervision due to the COVID-19 pandemic. Delimitations refer to the boundaries of a study, the scope within which the research will occur (Creswell & Creswell, 2018). This study focused on only three universities and a small sample. Further, eight participants were enrolled at University C, with only one participant from University A and one from University B. Other delimitations include the questions I asked, the focus on students’ experiences only, and using a single interview versus following the students longitudinally to examine their experience over time. Recommendations for Future Research This study assessed eight assumed KMO influences thought to impact trainee experiences as they complete the required supervision hours for their BCBA credential. The limited scope of this study afforded for inquiry at only three universities in the northeast, with only 10 students. Future research should focus on larger numbers of institutions and a more significant sample size. Additional suggestions for future research include investigating the need for intensive, ongoing supervision for newly-certified BCBAs, as the BCBA credential is designed to be an entry-level certification. Further, evaluating the impact prior experience working in applied behavior analysis may have on trainee experiences is an area worthy of consideration for study. Finally, future research could focus on the impact of differentiated supervision experiences. Trainee experiences may be influenced by the number and diversification of supervisors, settings, and client experiences, making this a recommended topic for future research. 90 Conclusion Clinical supervision is a formative experience for students pursuing the BCBA credential. Supervision facilitates a trainee’s clinical, professional, and ethical skill development and fosters the transition from student to practitioner. It is imperative that students have the knowledge and motivation to achieve this end and have the required support from the organizations within which trainees mature. This study used a gap analysis framework (Clark & Estes, 2008) to identify assumed influences that impact BCBA trainee experiences and success. This study’s findings confirmed five of eight assumed KMO influences and identified two emergent influences. These findings were analyzed and used to generate research-based recommendations designed to close KMO performance gaps. These recommendations were then used to create a plan for implementation and evaluation using the Kirkpatrick and Kirkpatrick (2016) New World Model. This project validated the important role student knowledge and motivation play in their BCBA trainee experience. BCBA trainees need additional support to understand the purpose of supervision, how to successfully navigate the administrative demands of the supervision process, and how to self-reflect and self-evaluate their own skill development. While students do align the supervision process with their personal identities and value the role it plays in their development as BCBAs, they require additional confidence in their ability to successfully transition from student to clinician. Further, trainees need to prioritize their clinical, professional, and ethical skill development over the accruement of experience hours. Finally, it is important that organizations provide students with consistent opportunities to participate in tasks aligned with the BCBA credential and offer ongoing and effective communication regarding the specifics of their supervisory experiences. 91 References American Psychological Association. (2015). Guidelines for clinical supervision in health service psychology. The American Psychologist, 70(1), 33–46. https://doi.org/10.1037/a0038112 Anderman, E., & Anderman, L. (2006). Attributions. Retrieved from http://www.education.com/reference/article/attribution-theory Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman. Association for Applied Behavior Analysis International. (n.d.) Verified course sequence program. https://www.abainternational.org/vcs/bacb.aspx Baer, D., Wolf, M., & Risley, T. (1968). Some defining dimensions of applied behavior analysis. Journal of Applied Behavior Analysis, 1(1), 91–97. https://doi.org/10.1901/jaba.1968.1- 91 Baker, L. (2006). Metacognition. Retrieved from http://www.education.com/reference/article/metacognition/September 14, 2016 Bandura, A. (1997). Self-efficacy: The exercise of control. W.H. Freeman. Bandura, A. (1994). Self-efficacy. In V.S. Ramachaudran (Ed.), Encyclopedia of human behavior (Vol. 4, pp. 71-81). Academic Press. (Reprinted in H. Friedman [Ed.], Encyclopedia of mental health. Academic Press, 1998). Barnett, J. (2014). Introduction: The (hopefully) essential primer on clinical supervision. Journal of Clinical Psychology, 70(11), 1023–1029. https://doi.org/10.1002/jclp.22123 92 Barnett, J., & Molzon, C. (2014). Clinical supervision of psychotherapy: Essential ethics issues for supervisors and supervisees. Journal of Clinical Psychology, 70(11), 1051–1061. https://doi.org/10.1002/jclp.22126 Behavior Analyst Certification Board. (n.d.-a). About behavior analysis. https://www.bacb.com/about-behavior-analysis/ Behavior Analyst Certification Board. (n.d.-b). About the BACB. https://www.bacb.com/about/#:~:text=%C2%AE,consumers%20of%20behavior%2Danal Behavior Analyst Certification Board. (n.d.-c). BACB certificant data. https://www.bacb.com/BACB-certificant-data Behavior Analyst Certification Board (n.d.-d). Board certified behavior analyst. https://www.bacb.com/bcba/ Behavior Analyst Certification Board. (n.d.-e). Supervision, assessment, training, and oversight. https://www.bacb.com/supervision-and-training/ Behavior Analyst Certification Board. (2005). BACB August 2005 newsletter. https://www.bacb.com/wp-content/uploads/2020/05/BACB_Newsletter_8_05.pdf Behavior Analyst Certification Board. (2011a). BACB January 2011 newsletter. https://www.bacb.com/wp-content/uploads/2020/05/BACB_Newsletter_01_2011.pdf Behavior Analyst Certification Board. (2011b). BACB September 2011 newsletter: Special edition on supervision. https://www.bacb.com/wpcontent/uploads/2020/05/BACB_ Newsletter_9-9-11.pdf Behavior Analyst Certification Board. (2012a). BACB December 2012 newsletter. https://www.bacb.com/wp-content/uploads/2020/05/BACB_Newsletter_12-12.pdf 93 Behavior Analyst Certification Board. (2012b). BACB January 2012 newsletter. https://www.bacb.com/wp-content/uploads/2020/05/BACB_Newsletter_101317-2.pdf Behavior Analyst Certification Board. (2013). BACB February 2013 newsletter: Special edition on new standards. https://www.bacb.com/wp-content/uploads/2020/05/ BACB_Newsletter_2-13.pdf Behavior Analyst Certification Board. (2014). BACB December 2014 newsletter: Special edition on BCBA supervision standards. https://www.bacb.com/wp-content/uploads/2020/05/ BACB_Newsletter_12-14.pdf Behavior Analyst Certification Board. (2017a). BACB January 2017 newsletter: Special edition on BCBA/BCaBA Task List (5th ed.). Retrieved from https://www.bacb.com/wp-content/ uploads/2020/05/January2017-newsletter-200828.pdf Behavior Analyst Certification Board. (2017b). BACB October 2017 newsletter: Special edition on experience and supervision requirements. https://www.bacb.com/wp- content/uploads/2020/05/BACB_Newsletter_101317-2.pdf Behavior Analyst Certification Board. (2017c). BCBA task list (5th ed.). https://www.bacb.com/ wp-content/uploads/2020/08/BCBA-task-list-5th-ed-210202.pdf Behavior Analyst Certification Board. (2018a). Consulting supervisor requirements for new BCBAs supervising fieldwork. https://www.bacb.com/wp-content/uploads/2020/11/ Consultation-Supervisor-Requirements-and-Documentation_201119-2.pdf Behavior Analyst Certification Board. (2018b). Supervisor training curriculum outline (2.0). https://www.bacb.com/wp-content/uploads/2020/05/Supervision_Training_Curriculum_ 190813.pdf 94 Behavior Analyst Certification Board. (2020a). BCBA 2022 eligibility requirements. https://www.bacb.com/wp-content/uploads/2020/11/BCBA-2022Eligibility Requirements_210212-2.pdf Behavior Analyst Certification Board. (2020b). Board certified behavior analyst handbook. https://www.bacb.com/wpcontent/uploads/2020/11/BCBAHandbook_201119.pdf Behavior Analyst Certification Board. (2019). US employment demand for behavior analysts: 2010-2018. https://www.bacb.com/wp-content/uploads/2020/05/US-Employment- Demand-for-Behavior-Analysts_2019.pdf Behavior Analyst Certification Board. (2021). US employment demand for behavior analysts: 2010-2020. https://www.bacb.com/wp-content/uploads/2021/01/BurningGlass2021_ 210126.pdf Bernard, J. M., & Goodyear, R. K. (2004). Fundamentals of clinical supervision (3rd ed.). Allyn & Bacon. Borders, D. (2014). Best practices in clinical supervision: Another step in delineating effective supervision practice. American Journal of Psychotherapy, 68(2), 151–162. https://doi.org/10.1176/appi.psychotherapy.2014.68.2.151 Clark, R., & Estes, F. (2008). Turning research into results: A guide to selecting the right performance goals. Information Age Publishing. Cole, M., Kolko, D., & Craddick, R. (1981). The quality and process of the internship experience. Professional Psychology, 12(5), 570–577. https://doi.org/10.1037/0735- 7028.12.5.570 95 Creaner, M. (2014). Current trends in clinical supervision: Introduction to the special section. Counselling Psychology Quarterly, 27(4), 325–333. https://doi.org/10.1080/09515070.2014.965904 Creswell, J., & Creswell, J. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). Sage Publications. Curtis, D. F., Elkins, S. R., Duran, P., & Venta, A. C. (2016). Promoting a climate of reflective practice and clinician self-efficacy in vertical supervision. Training and Education in Professional Psychology, 10(3), 133–140. https://doi.org/10.1037/tep0000121 Deochand, N., & Fuqua, R. (2016). BACB certification trends: State of the states (1999-2014). Behavior Analysis in Practice, 9, 243–252. https://doi.org/10.1007/s40617-016-0118-z DiGennaro Reed, F., & Henley, A. (2015). A survey of staff training and performance management practices: The good, the bad, and the ugly. Behavior Analysis in Practice, 8, 16–26. https://doi.org/10.1007/s40617-015-0044-5 Dixon, D., Linstead, E., Granpeesheh, D., Novack, M., French, R., Stevens, E., Stevens, L., & Powell, A. (2016). An evaluation of the impact of supervision intensity, supervisor qualifications, and caseload on outcomes in the treatment of autism spectrum disorder. Behavior Analysis in Practice, 9, 339–348. https://doi.org/10.1007/s40617-016-0132-1 Eccles, J. (2006). Expectancy value motivational theory. Retrieved from http://www.education.com/reference/article/expectancy-value-motivational-theory/. Eccles, J., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53(1), 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153 96 Ehrich, L., Hansford, B., & Tennent, L. (2004). Formal mentoring programs in education and other professions: A review of the literature. Educational Administration Quarterly, 40(4), 518–540. https://doi.org/10.1177/0013161X04267118 Elliot, A., Dweck, C., & Yeager, D. (2017). Handbook of competence and motivation (2nd ed.). Guilford Press. Ellis, M. V. (1991). Critical incidents in clinical supervision and in supervisor supervision: Assessing supervisory issues. Journal of Counseling Psychology, 38, 342–349. https://doi.org/10.1037/0022-0167.38.3.342 Ellis, M. V., Berger, L., Hanus, A. E., Ayala, E. E., Swords, B. A., & Siembor, M. (2014). Inadequate and harmful clinical supervision: Testing a revised framework and assessing occurrence. The Counseling Psychologist, 42, 434–472. https://doi.org/10.1177/0011000013508656 Ellis, M., Hutman, H., & Chapin, J. (2015). Reducing supervisee anxiety: Effects of role induction intervention for clinical supervision. Journal of Counseling Psychology, 62(4), 608–620. https://doi.org/10.1037/cou0000099 Falender, C. A., Shafranske, E. P., & Ofek, A. (2014). Competent clinical supervision: Emerging effective practices. Counselling Psychology Quarterly, 27(4), 393–408. https://doi.org/10.1080/09515070.2014.934785 Gallimore, R., & Goldenberg, C. (2001). Analyzing cultural models and settings to connect minority achievement and school improvement research. Educational Psychologist, 36(1), 45-56. https://doi.org/10.1207/S15326985EP3601_5 Garza, K., McGee, H., Schenk, Y., & Wiskirchen, R. (2018). Some tools for carrying out a proposed process for supervising experience hours for aspiring board certified behavior 97 analysts. Behavior Analysis in Practice, 11(1), 62–70. https://doi.org/10.1007/s40617- 017-0186-8 Gome, J., Paltridge, D., & Inder, W. (2008). Review of intern preparedness and education experiences in general medicine. Internal Medicine Journal, 38, 249–253. https://doi.org/10.1111/j.1445-5994.2007.01502.x Goodyear, R., & Bernard, J. (1998). Clinical supervision: Lessons from the literature. Counselor Education and Supervision, 38, 6–22. https://doi.org/10.1002/j.1556-6978.1998. tb00553.x Hajiaghamohseni, Z., Drasgow, E., & Wolfe, K. (2020) Supervision behaviors of board certified behavior analysts with trainees. Behavior Analysis in Practice, 14(1), 97-109. https://doi.org/10.1007/s40617-020-00492-1 Hartley, B., Courtney, W., Rosswurm, M., & LaMarca, V. (2016). The apprentice: An innovative approach to meet the behavior analysis certification board’s supervision standards. Behavior Analysis in Practice, 9(4), 329–338. https://doi.org/10.1007/s40617-016-0136-x Helvey, C., Thuman, E., & Cariveau, T. (2021). Recommended practices for individual supervision: Considerations for the behavior-analytic trainee. Behavior Analysis in Practice, 9(4). https://doi.org/10.1007/s40617-021-00557-9 Jones, A. (2006). Clinical supervision: What do we know and what do we need to know? A review and commentary. Journal of Nursing Management, 14, 577–585. https://doi.org/10.1111/j.1365-2934.2006.00716.x Kangos, K. A., Ellis, M. V., Berger, L., Corp, D. A., Hutman, H., Gibson, A., & Nicolas, A. I. (2018). American Psychological Association guidelines for clinical supervision: 98 Competency-based implications for supervisees. The Counseling Psychologist, 46(7), 821–845. https://doi.org/10.1177/0011000018807128 Kilminster, S., & Jolly, B. (2000). Effective supervision in clinical practice settings: A literature review. Medical Education, 34, 827–840. https://doi.org/10.1046/j.1365-2923.2000. 00758.x Kirkpatrick, J. & Kirkpatrick, W. (2016). Kirkpatrick’s four levels of training evaluation. Association for Talent Development. Krathwohl, D. (2002). A revision of Bloom’s taxonomy: An overview. Theory into Practice, 41, 212–218. https://doi.org/10.1207/s15430421tip4104_2 LeBlanc, L., & Luiselli, J. (2016). Refining supervisory practices in the field of behavior analysis: Introduction to the special section on supervision. Behavior Analysis in Practice, 9(4), 271–273. https://doi.org/10.1007/s40617-016-0156-6 LeBlanc, L., Heinicke, M., & Baker, J. (2012). Expanding the consumer base for behavior- analytic services: Meeting the needs of consumers. Behavior Analysis in Practice, 5(1), 4–14. https://doi.org/10.1007/BF03391813 Maxwell, J. (2013). Qualitative research design: An interactive approach (3rd ed.). Sage. Mayer, R. E. (2011). Applying the science of learning. Pearson Education. Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation (4 th ed.). Jossey-Bass. Miehls, D., Everett, J., Segal, C., & duBois, C. (2013). MSW students’ views of supervision: Factors contributing to satisfactory field experiences. The Clinical Supervisor, 32(1), 128–146. https://doi.org/10.1080/07325223.2013.782458 99 Normand, M., & Kohn, C. (2013). Don’t wag the dog: Extending the reach of applied behavior analysis. The Behavior Analyst, 36(1), 109–122. https://doi.org/10.1007/BF03392294 Nosik, M., & Grow, L. (2015). Prominent women in behavior analysis: An introduction. The Behavior Analyst, 38, 225–227. https://doi.org/10.1007/s40614-015-0032-7 O’Donovan, A., Halford, W. K., & Walters, B. (2011). Towards best practice supervision of clinical psychology trainees. Australian Psychologist, 46(2), 101–112. https://doi.org/10.1111/j.1742-9544.2011.00033.x Pajares, F. (2006). Self-efficacy theory. Retrieved from http://www.education.com/reference/article/self-efficacy-theory/. Parsons, M., & Rollyson, J. (2012). Evidence-based staff training: A guide for practitioners. Behavior Analysis in Practice, 5(2), 2-11. Patton, M. (2002). Qualitative research and evaluation methods (3rd ed.). Sage. Pillay, A., & Johnston, E. (2011). Intern clinical psychologists’ experiences of their training and internship placements. South African Journal of Psychology, 41(1), 74–82. https://doi.org/10.1177/008124631104100108 Pintrich, P. (2002). The role of metacognitive knowledge in learning, teaching, and assessing. Theory into Practice, 41(4), 219–225. https://doi.org/10.1207/s15430421tip4104_3 Pintrich, P. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95, 667–686. https://doi.org/10.1037/0022-0663.95.4.667 Ramos-Sánchez, L., Esnil, E., Goodwin, A., Riggs, S., Touster, L. O., Wright, L. K., Ratanasiripong, P., & Rodolfa, E. (2002). Negative supervisory events: Effects on 100 supervision and supervisory alliance. Professional Psychology, Research and Practice, 33(2), 197–202. https://doi.org/10.1037/0735-7028.33.2.197 Reichelt, S., & Skjerve, J. (2000). Supervision of inexperienced therapists: A quantitative analysis. The Clinical Supervisor, 19(2), 25–43. https://doi.org/10.1300/J001v19n02_02 Rønnestad, M. H., & Skovholt, T. M. (2013). The developing practitioner: Growth and stagnation of therapists and counselors. Routledge/Taylor & Francis Group. Rueda, R. (2011). The 3 dimensions of improving student performance: Finding the right solutions to the right problems. Teachers College Press. Schein, E.H. (2004). Organizational culture and leadership (3 rd ed.). Jossey-Bass. Schön, D. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. Jossey-Bass. Schraw, G., & McCrudden, M. (2006). Information processing theory. Retrieved from http://www.education/com/reference/article/information-processing-theory/ Sellers, T., Alai-Rosales, S., & MacDonald, R. (2016). Taking full responsibility: The ethics of supervision in behavior analytic practice. Behavior Analysis in Practice, 9, 299–308. https://doi.org/10.1007/s40617-016-0144-x Sellers, T., LeBlanc, L., & Valentino, A. (2016). Recommendations for detecting and addressing barriers to successful supervision. Behavior Analysis in Practice, 9(4), 309–319. https://doi.org/10.1007/s40617-016-0142-z Sellers, T., Valentino, A., & LeBlanc, L. (2016). Recommended practices for individual supervision of aspiring behavior analysts. Behavior Analysis in Practice, 9(4), 274–286. https://doi.org/10.1007/s40617-016-0110-7 101 Sellers, T., Valentino, A., Landon, T., & Aiello, S. (2019). Board certified behavior analysts’ supervisory practices of trainees: Survey results and recommendations. Behavior Analysis in Practice, 12(3), 536–546. https://doi.org/10.1007/s40617-019-00367-0 Sewell, K. (2018). Social work supervision of staff: A primer and scoping review. Clinical Social Work Journal, 46, 252–265. https://doi.org/10.1007/s10615-018-0679-0 Shepley, C., Allan Allday, R., & Shepley, S. (2017). Towards a meaningful analysis of behavior analyst preparation programs. Behavior Analysis in Practice, 11(1), 39–45. https://doi.org/10.1007/s40617-017-0193-9 Shook, G. (2005). An examination of the Integrity and future of the behavior analyst certification board credentials. Behavior Modification, 29(3), 562–574. https://doi.org/10.1177/0145445504274203 Shook, G., & Favell, J. (2008). The Behavior Analyst Certification Board and the profession of behavior analysis. Behavior Analysis in Practice, 1(1), 44–48. https://doi.org/10.1007/BF03391720 Shook, G., & Neisworth, J. (2005). Ensuring appropriate qualifications of applied behavior analyst professionals: The Behavior Analyst Certification Board. Exceptionality, 13(1), 3–10. https://doi.org/10.1207/s15327035ex1301_2 Tompkins, A. (2013). Enhancing self-efficacy to achieve competence. Journal of Cognitive Psychotherapy, 27(1), 71–80. https://doi.org/10.1891/0889-8391.27.1.71 Tugendrajch, S., Sheerin, K., Andrews, J., Reimers, R., Marriott, B., Cho, E., & Hawley, K. (2021). What is the evidence for supervision best practices? The Clinical Supervisor, 40, 68–87. Advance online publication. https://doi.org/10.1080/07325223.2021.1887785 102 Turner, L., Fischer, A., & Luiselli, J. (2016). Toward a competency-based, ethical, and socially valid approach to the supervision of applied behavior analytic trainees. Behavior Analysis in Practice, 9, 287–298. https://doi.org/10.1007/s40617-016-0121-4 Valentino, A., LeBlanc, L., & Sellers, T. (2016). The benefits of group supervision and a recommended structure for implementation. Behavior Analysis in Practice, 9, 320–328. https://doi.org/10.1007/s40617-016-0138-8 Wickham, J. (2012). Supervision: An issue for supervisors and supervisees. Advances in Psychiatric Treatment, 18, 416–417. https://doi.org/10.1192/apt.bp.111.009191 Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25, 68–81. https://doi.org/10.1006/ceps.1999.1015 Wigfield, A., Rosenzweig, E., & Eccles, J. (2017). Achievement values: Interactions, interventions, and future directions. In A. Elliot, C. Dweck, & D. Yeager (Eds.), Handbook of Competence & Motivation (2 nd ed., pp. 116-134). The Guilford Press. Yough, M., & Anderman, E. (2006). Goal orientation theory. Retrieved from http://www.education.com/reference/article/goal-orientation-theory/ 103 Appendix A: Definitions The following terms are used throughout the study and are important to develop an understanding of the concepts and ideas which are discussed therein. • BACB Supervisor Training Curriculum Outline: This outline provides a list of skills and tasks the supervisor should use to direct their practices while supervising students accruing experience hours (BACB, 2018b). • BCBA Supervisor: A BCBA who has completed the 8-hour supervision module and can oversee trainees as they complete their experience hours (Supervision and Training, n.d.) • Board Certified Behavior Analyst: a graduate-level certification in behavior analysis (BACB, 2020b). • Experience Hours: fieldwork hours completed under the supervision of the BCBA supervisor. The completion of these hours is required for trainees pursuing credentialing through the BACB (Hajiaghamohseni et al., 2020). • Supervision: A wide range of activities, including oversight of a trainee’s work, development of new skill sets, maintenance of previously learned skills, modeling data- based decision making, and setting an example for ethical behavior (LeBlanc & Luiselli, 2016). • Supervision contract: A written contract between the supervisor and supervisee designed to protect both parties. The contract should include the responsibilities of each party, descriptions of goals, objectives, and training activities to facilitate skill development for the trainee, as well as procedural safeguards including grounds for termination, ethical standards, document storage requirements (BACB, 2020a). 104 • Supervisory Alliance: The bond between supervisor and supervisee; grounded in trust, respect, communication, and goal alignment. (Bernard & Goodyear, 2004; Falender et al., 2014). • Trainee: A trainee is an individual who is accruing experience hours under a BCBA supervisor as they pursue certification as a BCBA (BACB, 2020b). 105 Appendix B: The Researcher I am a board certified behavior analyst and the founding executive director of an organization that employs BCBAs. I have worked in the fields of autism and applied behavior analysis for the past 20 years as both a direct service professional (e.g., teacher, behavior therapist) as well as a supervisor who was responsible for training and overseeing clinical staff. I provide ongoing coaching to the BCBAs who work at my organization and help them continue to develop their professional, leadership, and clinical skill repertoires. I have potential biases in that I have seen many newly-certified BCBAs begin their careers without effective clinical or professional training, and this influenced both my perspective and interest in this topic. I have seen the results of ineffective supervision and have mentored many BCBAs who have been poorly trained and expressed displeasure with their supervisory experience. Additionally, I am an alumna of one of the universities serving as a research site. I have an ongoing relationship with University A, and my company has employed students, both past and present, and provided them with BCBA supervision while accruing their required experience hours. To mitigate assumptions and biases, I reflected on my experience receiving BCBA supervision. As a prior student myself, I was aware of how I approached my supervision experience and my feelings surrounding it. I considered my experiences supervising others and how these may influence my thinking as a researcher. I recognized I have a connection personally and professionally with the universities and focused my attention on remaining neutral and without predisposition. I looked to identify assumptions or prejudices I may unknowingly have and remained keenly aware of these biases throughout my research. I considered them in the creation of my interview questions, their ordering, and delivery as well as 106 how they may have influenced the collection and interpretation of data (Merriam & Tisdell, 2016). While interviewing, I remained acutely aware of my biases as a former student, supervisor, and current BCBA and made every effort to remain neutral and not impart opinion or influence. 107 Appendix C: Instruments and Protocols This study utilized a brief survey to help identify potential participants. The results of the survey allowed me to identify students who were interested in participating and met criteria for the study. Once identified, I interviewed participants using semi-structured interviews and gathered relevant documents for document analysis. Pre-Screening Survey I administered a brief survey designed in the Qualtrics platform to gather basic demographic information about potential participants as well as assess their willingness to be interviewed and participate in the research. The results of the survey were reviewed, and I selected participants who met the inclusion and exclusion criteria and provided a balanced representation across age, gender, and supervisory experience. The following script was read to potential participants during one of their class sessions. The Qualtrics survey was then administered to all students for completion during or after class. Survey Script Good afternoon. My name is Carrie Kahana, and I am a board certified behavior analyst and currently a doctoral candidate at the University of Southern California. My area of interest is how to best prepare aspiring BCBAs for their future work in the field. My research is focused on supervision experiences, specifically from the lens of students who are accruing experience hours under a qualified BCBA supervisor. My study is designed to help me learn more about how KMO factors influence supervision experiences for students, and a key component of my research will be hearing from students like yourselves. Your participation will require a brief interview, which will take no more than an hour, and will be scheduled at your convenience and take place via Zoom. Additionally, you will be asked to share documents related to your 108 supervision experience including your supervision contract and any handbooks or supervision documents your supervisor or clinical placement provided you. I will ask you to first redact these documents by removing any identifying information to protect both your, your supervisor’s, your university’s, and your placement’s anonymity. I will use the information gathered in these interviews to help me make recommendations for ways we can improve BCBA supervision experiences for trainees. By participating in this study, you will have the opportunity to share your experiences and perspective and promote positive change for future students and the field of behavior analysis. Your professor will now email each of you a brief survey to gather basic demographic information and assesses your willingness to participate in my research. Thank you for your consideration! Written Introduction to Survey My study is designed to learn about the knowledge, motivation, and organizational factors that influence student’s BCBA supervision experiences. The information gathered through this research will be used to generate recommendations on ways to improve trainee experiences. This brief survey is designed to gather basic demographic information and assess your interest in participating in my research. Your participation is this study is completely voluntary. Thank you for time and for the opportunity to learn from you. Carrie Kahana, MA, BCBA, LBA (NY) Doctoral Candidate, University of Southern California ckahana@usc.edu 109 Survey Questions 1. Age: How old are you? 2. Gender: How do you identify? a. Male b. Female c. Non-binary/third gender d. Other ________ e. Prefer not to say 3. Ethnicity: How would you describe yourself? a. White b. Black or African American c. American Indian or Alaska Native d. Asian e. Native Hawaiian or Pacific Islander f. Other _________ g. Prefer not to say 4. Education: What is your highest level of education? a. High school graduate b. Some college c. 2-year degree d. 4-year degree e. Master’s degree f. Doctorate 110 5. Experience: Are you currently accruing supervision hours under a qualified BCBA supervisor? (If “c” is selected, survey will skip to Question 8) a. Yes b. No, I haven’t begun yet c. No, I have already completed accruing all my experience hours 6. Experience: Have you accrued more than 100 experience hours thus far? (This question will only be displayed if student selects “a” for Question 5) a. Yes b. No c. I’m not sure 7. Experience: Approximately how many supervision hours have you accrued? (This question will only be displayed if student selects “a” for Question 6) a. 100–499 b. 500–999 c. 1,000–1,499 d. 1,500 + 8. Participation: Would you be willing to participate in this study (e.g., participate in a short interview about your supervision experience as a BCBA trainee and share relevant supervisory documentation)? (This question will only be displayed if student selects “c” for Question 5 or “a” for Question 6) a. Yes (If yes, the following two questions will be displayed). b. No 9. Please indicate your name below: 111 10. Please indicate your email address below: Interview Protocol Each interview consisted of a brief introduction, a series of questions designed to highlight knowledge, motivation, and organizational influences, and a wrap up where the interview was concluded. Introduction to the Interview This was read to the interviewee prior to beginning the interview: Good afternoon, (Name). My name is Carrie Kahana and I am a doctoral student at the University of Southern California. I am currently looking to learn more about BCBA supervision experiences, specifically through the lens of students. Your experience and insight will help me understand more about student, or trainee, experiences and how universities, employers, and supervisors can best support students. Everything you share today will be kept confidential. Before we begin, I want to share and review the study protocol and consent form with you. I will place these documents in the chat now. <Review documents with interviewee> Do you have any questions about either of these documents? Do you consent to participate in this research? <Get vocal confirmation from student>. I would like to record our discussion today so I can review and make sure I am accurately capturing your experiences and opinions. Do I have your consent to do so? Before we get started, do you have any questions I can answer for you? Background/Demographic Information 1. What program are you enrolled in at the university? 2. When did you begin your coursework at the university? 3. How many classes have you taken thus far? 112 4. When did you begin accruing supervision hours? 5. Are you completing concentrated fieldwork hours (1,500) or supervised fieldwork hours (2000)? 6. Are you completing your experience hours as part of a university practicum? 7. How many supervision hours have you accrued thus far? 8. In what type of setting are you accruing your experience hours? (E.g. Center-based, school program, home program) 9. How did you go about finding your supervisor? Knowledge Influences Conceptual 10. What are you hoping to gain from your supervision experience? a. What do you expect to learn? 11. What do you understand to be the goals of supervision? 12. What are the components of effective supervision? 13. Why are BCBA students supervised? b. How do you know this? Procedural 14. Describe the process of supervision. a. Walk me through a typical supervision session. 15. What is the role of a BCBA supervisor? a. What does this look like? b. What types of things is your supervisor responsible for within your supervision experience? 113 16. What is the role of a BCBA trainee? a. What does this look like? b. What types of things are you responsible for within your supervision experience? c. How do you know this? 17. What are your supervisor’s expectations of you? Metacognitive 18. Describe your process for self-evaluating your own skill development. a. How do you identify areas of strength or weakness? 19. Reflect on a time when you did not know how to execute a skill with your client. What did you do? 20. If there was something you wanted more training on or didn’t understand, how would you go about addressing it? a. Can you give me an example of a time you did this? Motivation Influences Self-Efficacy 21. How did you feel as you began your supervision experience? a. Did you feel prepared to take full advantage of your supervision experience? Why or why not? 22. How confident are you in your ability to transfer what you are learning in your coursework into practice with clients? a. How has this changed since you began the supervision process? 23. Do you believe you will continue to grow as a clinician despite setbacks or challenges, whether personal or professional? 114 a. Why or why not? Utility Value 24. Is your supervision experience useful to you? a. What makes you say that? 25. What part of your supervision experience has been the most valuable to you? a. What makes you say that? 26. What effect, if any, does your supervisor have on your future as a BCBA? a. What makes you say that? Attainment Value 27. As a future BCBA, how important is the supervision experience? a. What makes you say that? Goal Orientation 28. What is your primary goal as you move through your supervision experience? a. How has that goal changed as you have progressed through your supervision? 29. Think of yourself as a clinician prior to beginning your experience hours with your supervisor. In what ways, if any, have you changed? a. Areas of growth? b. Areas to continue to target? 30. Describe why one of the following two statements best fits you: the first has you accruing all your hours within the targeted time frame, allowing you to sit for your qualifying exam ‘on time;’ the second has you taking longer to accrue your hours but leads to a deeper understanding of content and application. 115 Organizational Influences Cultural Setting 31. Think back to when you first began working with your supervisor. What did the process look like when you entered the supervisory relationship? a. How, if at all, did your supervisor (or organization) set the stage for the supervisory relationship? 32. How, if it all, was the supervision process explained to you? a. Was this helpful? How did it make you feel? 33. Did you know what your supervision experience would look like? For example, the structure of supervision, activities that will take place during supervision, frequency of same, roles and expectations, and process for feedback and evaluation. a. How did you know this? 34. What, if anything, could your supervisor or employer have done to better facilitate the beginning the supervisory experience? 35. What would the ideal supervision experience look like, from beginning to end? Cultural Models 36. Tell me about your relationship with your supervisor. a. What are the best aspects of the relationship? b. What are the most challenging aspects of the relationship? 37. Do you feel that your supervisor is a good fit for you? a. What makes you say that? 38. Do you feel that your supervisor listens to you and understands your goals and needs? a. What makes you say that? 116 b. Can you give me an example? 39. What, if anything, would make the relationship better? Conclusion to the Interview This was read at the end of each interview: This concludes the formal interview. Before we wrap up, is there anything you would like to add regarding supervision, or any of the topics we discussed today? Is there anything you would like to amend or retract? Thank you for taking the time to meet with me today and sharing your experience as a BCBA trainee. As discussed, all your responses will remain confidential. If there are any additional points you would like to share or should you have any questions, you can reach out to me at any time. To help me gain a fuller understanding of your supervision experience, I would also like to review documents that have guided your supervision experience. Specifically, any course syllabi related supervision, your supervision contract, and any supervision handbooks or supervision guidelines provided to you by your supervisor or clinical placement as you began your supervision experience. I will send a follow-up email detailing these items as well as requirement for sharing. Your confidentiality and anonymity are paramount so please be sure that all identifying information has been removed before sharing to maintain anonymity, I kindly ask you black out or remove any identifying information from the document before sharing. This includes your name, your supervisor’s name, the name of your university, and the name of the clinical placement. Do you have any questions about this? To facilitate document sharing I have created a folder for you on my university-affiliated Google Drive. I will put the link in our chat now but also send it to you in my follow-up email. Do you have any questions about this? 117 Thank you again for your participation in this research and your impact on the current and future fields of behavior analysis! Document Request This email was sent to participants following the semi-structured interview. Dear Participant, Thank you for participating in my study and sharing information about your supervision experience with me. To gain a fuller understanding of your experience, I would like to review documentation that you have received while accruing your experience hours under your BCBA supervisor. These documents include your supervision contract, any handbooks or guidelines for your supervision experience provided to you by your supervisor, clinical placement, or university, and course syllabi related to your clinical supervision experience. I kindly ask you email me these documents in their redacted form, as it is critical to ensure anonymity. Kindly black out or remove any identifying information including your name, your supervisor’s name, and the name of your clinical placement or university. I kindly ask you upload these documents into the linked Google Drive folder by (date) so I have the opportunity to review them and incorporate this information into my analysis and recommendations. Thank you for your support and participating in this research. You are making a difference to the future or our field! Sincerely, Carrie Kahana University of Southern California 118 Participant Follow-Up I followed up with each participant after interviews and document analysis had been completed. I thanked them for their participation in this research study and for sharing their supervision experiences with me. 119 Appendix D: Ethics A study’s validity and reliability are unequivocally linked to the researcher’s ethics (Merriam & Tisdell, 2016). As the primary research instrument, I prioritized high ethical standards throughout my study and while conducting all research. The study was submitted to the institutional review board (IRB) at the University of Southern California for review and observed all guidelines set forth by the IRB. The IRB assessments allowed me to confirm adherence to ethical guidelines and ensure no harm was caused to participants. While conducting the screening survey, interviews, and document analysis, I informed all participants of the study’s purpose and methodology. Participants received informed consent forms at the onset of the study and were made aware of the voluntary nature of their participation, their ability to withdraw without penalty at any time, and protocols to provide anonymity and protect the confidentiality of the information they shared (Merriam & Tisdell, 2016). I requested permission to record all interviews prior to doing so and made participants aware of their right to not answer questions or terminate at any time. I ensured all documents were redacted before analysis to ensure anonymity for all parties. Data collected via interviews and document analysis was stored in a folder in a password-protected, dual authentication cloud- based server (USC’s Google Drive). 120 Appendix E: Theoretical Framework Alignment Matrix Research question Theoretical framework Data instrument questions What are the knowledge factors affecting student experiences while accruing supervision hours? Gap analysis framework (Clark & Estes, 2008) Interview Questions 10–20 Document analysis What are the motivation influences affecting student experiences while accruing supervision hours? Gap analysis framework (Clark & Estes, 2008) Interview Questions 21–30 What are the organizational influences affecting student experiences while accruing supervision hours? Gap analysis framework (Clark & Estes, 2008) Interview Questions 31–39 Document analysis Demographic questions Pre-Survey Screening Questions 1–9 121 Appendix F: Consent Form Carrie A. Kahana University of Southern California INFORMED CONSENT FOR NON-MEDICAL RESEARCH Study Title: Maximizing BCBA Trainee Development During Supervision: Knowledge, Motivation, and Organizational Influences Principal Investigator: Carrie A. Kahana Department: USC Rossier School of Education Introduction You are invited to take part in a research study conducted by Carrie Kahana at the University of Southern California. Please read through this form and ask any questions you may have before deciding whether you would like to participate. If you decide to participate, you will be asked to sign this form. A copy of the signed form will be provided to you for your records. Key Information The following is a short summary of this study to help you decide whether you should participate. More detailed information is listed later in this form. 1. Being in this research study is voluntary–it is your choice. 2. You are being asked to take part in this study because you are a student at a university where the research is taking place. The purpose of this study is to learn about supervision experiences for aspiring board certified behavior analysts. Your participation in this study will last approximately 60 minutes for a one-on-one interview conducted via Zoom and the time it takes for you to share pertinent documents related to your supervision experience. Procedures will include completing a pre-interview screening that asks for some demographic information (e.g., name, 122 age, gender, ethnicity, level of education, etc.), an interview where you will have the opportunity to share your experiences with and thoughts on supervision, and document sharing via a secure Google Drive folder. If you decide to participate in the one-on-one interview conducted via Zoom, you will be contacted to schedule a time that is convenient for you. 3. There are minimal risks to the participant. More detailed information about the risks of this study can be found under the “Risk and Discomfort” section. There are no monetary or tangible benefits to participating in this research, however, your participation in this study may help us learn how to better prepare and support trainees accruing experience hours under BCBA supervisors. 4. If you decide not to participate in this research, there are no consequences to cancelling or withdrawing at any time Detailed Information Purpose The purpose of this study is to learn about supervision experiences from students who are accruing or have accrued the required supervision hours to become certified as a BCBA. We hope to learn how knowledge, motivation, and organizational influences can help maximize trainee development. You are invited as a possible participant because you are a student at the university, you are enrolled in behavior-analytic coursework, and you are currently being supervised by or have completed your experience hours under a qualified BCBA supervisor. About 15 participants will participate in this study. Procedures The following steps will be taken if you decide to participate in this study. 123 1. Pre-interview Screening: All students will be asked to complete a brief survey to gather demographic information and ascertain eligibility and interest in participating in the research. If interested, individuals will indicate by sharing their name and email address. 2. Interview: Participants will be contacted to schedule an interview via Zoom. Interviews will last approximately 60 minutes. The researcher will ask approximately 39 questions designed to learn about the participants supervision experience. The researcher will record the interview, with participant permission, and take notes throughout to facilitate data collection. 3. Document Sharing: Participants will be asked to share documentation from their supervision experiences. These redacted documents may include any or all the following: supervision contract, supervision documents shared by their supervision site or supervisor, supervision handbook provided by the clinical placement or supervisor, supervision guidelines provided by the university, and course syllabi related to clinical supervision. Risks and Discomforts Possible risks and discomforts you could experience during this study include negative emotions or other unanticipated feelings. Benefits There are no direct benefits to you from taking part in this study. However, your participation in this study may help us learn more about trainee experiences while accruing the experience hours required for BCBA certification. This information will allow the 124 researcher to make recommendations designed to better prepare students for their supervision experience and in turn, provide more meaningful outcomes. Privacy/Confidentiality We will keep your records for this study confidential as far as permitted by law. However, if we are required to do so by law, we will disclose confidential information about you. Efforts will be made to limit the use and disclosure of your personal information, including research study records, to people who are required to review this information. We may publish the information from this study in journals or present it at meetings. If we do, we will not use your name or the name of your university, placement, or supervisor. The University of Southern California’s Institutional Review Board (IRB) and Human Subject’s Protections Program (HSPP) may review study records. Alternatives If you wish to participate in this study but are uncomfortable with the interview process, a copy of the interview questions can be provided for completion at your convenience. Payments/Compensation You will not be compensated for your participation in this research. Voluntary Participation It is your choice whether to participate. If you choose to participate, you may change your mind and leave the study at any time. Withdrawal From Study Instructions Your participation in this research is voluntary and you are free to withdraw or refuse to answer any question at any time without negative consequence to you. If you wish to withdraw after scheduling an interview, you may do so by emailing the word “withdraw” with your first 125 and last name to the researcher, Carrie Kahana at ckahana@usc.edu. You need not provide a reason or rationale for withdrawing from the study. Contact Information If you have any questions or concerns about the research, please contact Carrie Kahana at ckahana@usc.edu This research has been reviewed by the USC Institutional Review Board (IRB). The IRB is a research review board that reviews and monitors research studies to protect the rights and welfare of research participants. Contact the IRB if you have questions about your rights as a research participant or have complaints about the research. You may contact the IRB at (323) 442-0114 or by email at irb@usc.edu. Statement of Consent I have read (or someone has read to me) the information provided above. I have been given a chance to ask questions. All my questions have been answered. By signing this form, I am agreeing to take part in this study. Name of Research Participant Signature Date Signed 126 Appendix G: Coding Sheet with Themes Knowledge Influences Conceptual knowledge Procedural knowledge Metacognitive knowledge BCBA skill repertoire Components of effective supervision Help-seeking Develop competency Paperwork confusion Self-reflection Theory to practice Self-advocacy Supervisee expectations Administrative/professional Areas of need; question-asking Feedback; model supervisor Skill development Competency development Information-sharing Mentor Provide opportunities Motivation Influences Self-efficacy Expectancy value Goal orientation Clinical skills/direct Supervisor impact Clinical development/growth throughout process Paperwork/indirect Value of supervision Hours accruement/paperwork Feelings at supervision onset Theory to practice Trainee goal for supervision Organizational Influences Cultural Setting Cultural Models Not doing BCBA work Collaboration/alignment Requirements/Paperwork Guidance/support Supervision procedures Knowledge/experience Culture Relationship/rapport Evaluation/Feedback 127 Appendix H: Implementation and Evaluation Plan The implementation and evaluation plan described in this study is based on the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016). Kirkpatrick and Kirkpatrick (2016) emphasize the importance of evaluating training and improvement programs to ensure they are effective, have value, and promote change and goal achievement (Kirkpatrick & Kirkpatrick, 2016). The New World Kirkpatrick Model consists of four levels of training and evaluation that are executed in reverse order: (4) Results, (3) Behavior, (2) Learning, and (1) Reaction (Kirkpatrick & Kirkpatrick, 2016). Level 4 determines the effect of learning on the organization and if the desired outcome has been achieved. Level 3 examines the behavior changes that occur as the result of training, specifically examining critical behaviors that individuals must e demonstrate to achieve the targeted results. Level 2 assesses participants to determine what they have learned and identifies what knowledge, skills, and attitudes individuals have toward the desired behavior. Level 1 evaluates an individual’s reaction to the training program and includes appraisals of learner engagement and satisfaction. This plan will describe each level of the New World Kirkpatrick Model (Kirkpatrick & Kirkpatrick, 2016), and detail recommendations to promote learning and program evaluation for BCBA trainees. The stakeholders for this study were students currently enrolled in BCBA coursework and accruing the required experience hours under a qualified BCBA supervisor. This study sought to identify the knowledge, motivation, and organizational influences that impact BCBA trainee experiences during supervision. Participants attended one of three universities in the northeast United States but worked in a variety of organizational settings. While the BACB’s goal for supervision is to provide students with consistent and meaningful experiences that will promote their clinical, professional, and ethical skills, this study did not focus on or evaluate any 128 one organization. Accordingly, the recommendations and the implementation and evaluation plan will focus broadly on solutions for students, supervisors, and organizations associated with BCBA trainee supervision experiences. Further, although this study’s recommendations cover gaps in student knowledge and motivation as well as organizational barriers, the focus of this implementation and evaluation plan is on those factors that students can affect. This is not to say that students cannot affect elements within cultural models and settings, but this study looked at organizational barriers from a student perspective. Level 4: Results and Leading Indicators Creating detailed metrics and methods for measuring the internal and external outcomes is essential for achieving organizational goals (Clark & Estes, 2008; Kirkpatrick & Kirkpatrick, 2016). As such, Table H1 outlines the desired outcomes, metrics, and methods for employers and supervisors who support BCBA trainees during their supervision experiences. The internal and external outcomes are grounded in this study’s recommendations to close validated gaps in knowledge, motivation, and organizational influences. 129 Table H1 Outcomes, Metrics, and Methods for External and Internal Outcomes Outcome Metric(s) Method(s) Internal outcomes Increase trainee knowledge of the purpose of supervision Results from written and vocal assessment completed by trainees following initial orientation training Review results from assessment and discussion to confirm trainee understanding Increase trainee knowledge of BACB supervision documentation processes and protocols Accurate completion of trainee documentation Monthly supervisor review of required forms for accuracy Increase trainee use of metacognition and self- reflection Trainee’s self-disclosed use of metacognitive strategies Quarterly discussions with trainees to assess their self-described use of metacognition Increase trainee self-efficacy related to connecting theory to practice Trainee’s self-disclosed level of efficacy connecting theory to practice Quarterly discussions with trainees to assess their self-described levels of efficacy Increase trainee awareness of goal orientation and the value of a mastery mindset Trainee’s self-disclosed level of awareness of goal orientation and value of a mastery mindset Quarterly discussions with trainees to assess their self-described value and implementation of a mastery mindset Improve systems for disseminating supervision procedures to trainees Number of trainees who demonstrate understanding of impending supervision process Trainee survey Increase opportunities for trainees to consistently participate in BCBA tasks Number of BCBA tasks completed by trainees each quarter Quarterly review of the number of indirect hours accrued External outcomes BACB adds content to address identified gaps in knowledge and motivation influences to required coursework sequence Number of hours required for students to complete Review content areas and hours required for BCBA coursework on BACB website 130 Level 3: Behavior Level 3 measures the degree to which individuals have changed their behavior because of the training (Kirkpatrick & Kirkpatrick, 2016). This section identifies critical behaviors and evaluation strategies, required drivers to support trainee’s application of critical behaviors, and organizational support considerations to maximize trainee success. Critical Behavior Students accruing the required experience hours for the BCBA credential are the stakeholders. Level 3 identifies critical behaviors, meaning behaviors individuals will need to consistently perform to promote the desired outcome (Kirkpatrick & Kirkpatrick, 2016). Critical behaviors identify actions trainees must engage in to maximize their BCBA supervision experiences and integrate knowledge and motivation influences identified in this study. The first critical behavior is that trainees demonstrate the ability to complete the required BACB documentation. The second critical behavior is for trainees to engage in self-reflection and identify areas in need of further development. The third critical behavior is trainees demonstrate self-efficacy and believe in their ability to connect theory to practice. The fourth critical behavior is students set goals grounded in a mastery mindset and based on self-evaluation. The fifth critical behavior is that trainees need to actively recruit opportunities to participate in tasks aligned with the BCBA credential. Finally, the sixth critical behavior is that trainees need to fully understand their impending supervision experience. The metrics, methods, and timing for each critical behavior are detailed in Table H2. 131 Table H2 Critical Behaviors, Metrics, Methods, and Timing for Evaluation Critical behavior Metric Methods Timing Trainees will correctly complete required BACB documentation The number of students who complete monthly forms with fewer than two errors Review completed forms Monthly Trainees will engage in self- reflection and identify areas of need Number of skills achieved and identified via reflection Monthly self- reflection report Monthly Trainees will demonstrate self- efficacy and confidence in ability to connect theory to practice Scores on self-efficacy measure Use existing self- efficacy measure to assess trainee’s perception and confidence Quarterly Trainees will set goals aligned to personal growth and based on self-evaluation The number of goals aligned with a mastery vs. performance mindset Goals will be written by trainees and reviewed by supervisor Quarterly Trainees will actively seek out opportunities to complete tasks aligned with the BCBA credential Number of times a trainee asks for supervision or practice in a specific skill area Review of number of requests made by the trainee Quarterly Trainees will demonstrate understanding of the structure of supervision and processes that will be used throughout Degree of trainee understanding after explanations during onboarding and orientation processes Interview to assess After first 90 days with the organization Required Drivers Trainees will require support from supervisors and organizations to successfully engage in critical behaviors. Required drivers provide support and accountability by reinforcing, monitoring, encouraging, and rewarding critical behaviors (Kirkpatrick & Kirkpatrick, 2016). 132 Table H3 identifies the required drivers to support critical behaviors and timing for implementation. Table H3 Required Drivers to Support Critical Behaviors Methods Timing Critical behaviors supported Reinforcing Hold new trainee orientation meeting. Provide training on the purpose of supervision, information on the structure and schedule for trainee’s supervision experience, and BACB documentation requirements and process for completion At onset of new trainee experience 1, 6 Job aid that provides a list of supervision activities, structure of activities, how to prepare for each type of activity, and a schedule of supervision sessions for upcoming quarter On onset of new trainee experience 6 Job aid that contains definitions and key terms found in BACB supervision documents. Ongoing 1 Job aid that provides a flow chart on supervision document completion. Ongoing 1 Job aid that provides a check list on supervision document completion Ongoing 1 Hold supervision group meeting for trainees targeting learning and motivation topics (e.g., metacognition, self-efficacy, mastery mindset); Provide training and ongoing opportunities for modeling, practice, coaching, and feedback Monthly 2, 3, 4 Job aids detailing strategies for metacognition and mastery mindset goal-setting Ongoing 2, 4 Provide reminders to trainees to recruit opportunities for BCBA tasks aligned with their goals and identified areas of need Ongoing 2, 5 Encouraging 133 Methods Timing Critical behaviors supported Supervisors model and provide opportunities for practice completing BACB forms and documentation Ongoing 1 Supervisors provide modeling and coaching on self-reflection and behavior aligned with a mastery mindset Ongoing 2, 4 Trainees meet with supervisors to engage in self- reflection and goal development Ongoing 2, 4 Supervisors model target skills for trainees allowing students to observe successful execution Ongoing 3 Supervisors model learning from their own mistakes and viewing mistakes as opportunities Ongoing 4 Supervisors model asking questions if they are unsure or require additional information Ongoing 1, 4, 6 Rewarding Supervisors provide trainees with positive feedback on accurate BACB form completion Ongoing 1, 3 Supervisors provide trainees with positive feedback on incremental growth Ongoing 2, 3, 4 Supervisors provide trainees with positive feedback on self-reflective behavior Ongoing 2, 3 Supervisors provide trainees with positive feedback when embracing growth mindset Ongoing 4 Supervisors provide trainees with positive feedback when they recruit BCBA experiences Ongoing 5 Public recognition for trainee’s success and growth within Supervision Group Ongoing 1, 2, 3, 4, 5 Monitoring Supervisor-trainee interaction within supervision sessions Ongoing 1, 2, 3, 4, 5, 6 Supervisor-trainee interaction within monthly supervision group meetings Monthly 1, 2, 3, 4, 5, 6 Review of completed BACB forms Monthly 1 Self-evaluation completed by trainee assessing achievements, challenges, goals, reflections, and questions Monthly 1, 2, 3, 4, 5, 6 134 Organizational Support Organizations, employers, and supervisors need to provide BCBA trainees with the supports and resources to realize success. Organizations must provide training to students to ensure they have the necessary knowledge and motivation to maximize their supervisory experience (Clark & Estes, 2008; Goodyear & Bernard, 1998; Kangos et al., 2018; Tompkins, 2013). To do this, organizations need to ensure supervisors have sufficient training in the identified knowledge and motivational influences so that they can provide the necessary support to trainees (APA, 2015; Barnett & Molzon, 2014; Goodyear & Bernard, 1998; O’Donovan et al., 2011). Finally, it is important that organizations and supervisors prioritize trainees’ skill development and recognize the importance of the supervisor’s impact on a student’s future as a BCBA. Level 2: Learning Level 2 measures how much participants improved their knowledge and skills because of the training provided (Kirkpatrick & Kirkpatrick, 2016). This section identifies learning goals trainees should achieve as a result of their participation in the training. Learning Goals Following the implementation of the proposed solutions, trainees will be able to: 1. Recognize the purpose of BCBA trainee supervision (Conceptual knowledge). 2. Demonstrate proficiency in completing required BACB paperwork and documentation (Procedural knowledge). 3. Reflect on their own learning and identify areas of need (Metacognitive knowledge). 4. Be confident in their ability to connect theory to practice (Self-efficacy). 135 5. Possess a mastery mindset in their approach to their own learning and development (Goal orientation). Program Learning goals will increase trainees’ knowledge and motivation and will be addressed through training, job aids, and ongoing opportunities for practice and feedback delivery. Upon inception of a trainee’s supervision experience, they will participate in a full-day orientation session designed introduce the trainee to their supervision experience and provide information needed for success. This training targets three main content areas: the importance of supervision, documenting the supervision experience, and motivational influences and supports. First, students will learn about the purpose of supervision and specifics related to their impending supervision experience, including schedule information and requirements for supervision activities. Next, trainees will receive information related to completing required BACB paperwork, including definitions, relevant documents, and processes for completing required paperwork. Trainees will have opportunities to observe paperwork completion and practice completing it with mock forms. Trainees will receive job aids detailing their supervision schedule, supervisory expectations, and completion of BACB documentation. Finally, trainees will learn about knowledge and motivational concepts, including metacognition, self-efficacy, and mastery mindset. Job aids for each influence type will be provided to facilitate learning and remembering information. Since students may begin accruing experience hours at different times, these orientation sessions will occur on a rolling basis and be scheduled individually and as needed. After the orientation session, trainees will join the organization’s supervision group, which includes all students currently accruing experience hours and led by a senior-level BCBA. 136 This supervision group will provide ongoing training on topics related to knowledge and motivation influences and provide consistent opportunities for discussion, modeling, coaching, and feedback delivery. These ongoing meetings allow trainees to observe BCBAs engaging in the target behaviors while learning alongside peers. Moreover, trainees have the chance to integrate the concepts they are learning is a safe and supportive environment. Individual supervision sessions should foster the supervisory alliance, provide trainees with opportunities to observe seasoned BCBAs completing target tasks, and offer scaffolded opportunities designed to promote self-efficacy and focus on skill development. Supervisors and organizations should work to create a safe learning environment where trainees are comfortable during the learning process are mistakes are viewed as opportunities for growth versus failures. Components of Learning Kirkpatrick and Kirkpatrick (2016) identified five components of learning: knowledge, skills, attitude, confidence, and commitment. Trainees accruing experience hours under a qualified BCBA supervisor need to have conceptual knowledge about the purpose of supervision, procedural knowledge relating to completing the required BACB paperwork, and metacognitive knowledge to continually self-evaluate their progress. Trainees need to be confident in their ability to transition from student to practitioner, embrace a mastery mindset throughout their development, and take an active role in recruiting opportunities to participate in tasks associated with the BCBA credential. 137 Table G4 Evaluation of the Components of Learning for the Program Methods or activities Timing Declarative knowledge “I know it.” Knowledge checks during orientation session During orientation session Pre- and post-training assessments After orientation session Knowledge checks during group supervision sessions During ongoing group supervision sessions Procedural skills “I can do it right now” Successful completion of mock BACB forms During ongoing group supervision sessions Role playing asking for BCBA task opportunities During ongoing group supervision sessions Attitude “I believe this is worthwhile” Discussions during orientation session During orientation session Pre- and post-training assessments After orientation session Discussion during group supervision following modeling and demonstration During ongoing group supervision sessions Supervisor observations of trainee behavior and commenting During ongoing individual supervision sessions Confidence “I think I can do it on the job” Discussions during orientation session During orientation session Pre- and post-training assessments After orientation session Discussions during group supervisions following practice and feedback During ongoing group supervision sessions Commitment “I will do it on the job” Discussions during orientation session During orientation session Pre- and post-training assessments After orientation session Individual goals developed by trainee and supervisor During ongoing group supervision sessions 138 Level 1: Reaction Level 1 assesses participants’ responses to the training and measures engagement, relevance, and satisfaction (Kirkpatrick & Kirkpatrick, 2016). Level 1 evaluations are both formative and summative and should take place both during the program to allow for any necessary adjustments and after training programs to help identify areas in need of improvement or adjustment. Table 15 identifies the methods used to evaluate trainees’ reactions to the program and levels of engagement. Table H5 Components to Measure Reactions to the Program Methods or Tools Timing Engagement Instructor observation During orientation sessions End-of-training survey End of orientation Observations during group supervision sessions Ongoing Relevance Instructor observation During orientation session End-of-training survey End of orientation Discussions with trainees during group supervision sessions Ongoing Customer Satisfaction Instructor observation During orientation session End-of-training survey End of orientation session Discussions with trainees during group supervision sessions Ongoing 139 Evaluation Tools Tools can be used at different stages of the training to evaluate the program’s effectiveness. This section includes options for both immediate and delayed evaluation. Immediately Following Program Implementation Throughout the orientation training, instructors will observe students to determine levels of engagement. Immediately following the orientation session, trainees will complete a survey designed to measure their satisfaction, relevance, and value for the training. Table 16 highlights survey questions. Table H6 Immediate Feedback Survey Strongly Disagree Disagree Neutral Agree Strongly Agree The training was relevant to me. I found the training useful. I found the training interesting. I understand the purpose of BCBA supervision. I understand the definitions and processes for completing BCBA paperwork. I am confident in my ability to complete BCBA paperwork. I understand what my supervision experience will look like. 140 Delayed for a Period After the Program Implementation The nature of supervision affords the opportunity for ongoing training and development. Students will participate in group supervision sessions throughout their supervision experience, so an additional survey will be distributed 90-days after the orientation training to gather data on student progress over time and assess the effectiveness of training and program implementation. A copy of this survey is found in Table 17. 141 Table H7 Delayed Feedback Survey Strongly disagree Disagree Neutral Agree Strongly agree I understand and can complete the required BCBA paperwork for supervision documentation. I engage in self-reflection and can identify areas I need further training. I understand and can employ a mastery mindset. I am confident in my ability to connect theory to practice. I believe the training received during group supervision is valuable. I believe the training received during group supervision is interesting. I believe the training received during group supervision is relevant to me. Open-ended questions What part of your training has been most helpful? List 2 things you have learned that help you succeed as a BCBA trainee? Data Analysis and Reporting Level 4 goals of this plan are increased accuracy of BACB form completion, trainee engagement in self-reflection, trainee understanding and demonstration of self-efficacy and mastery mindset, active recruitment of BCBA task opportunities by trainees, and improved 142 systems for detailing the supervision process to new trainees. Numerous metrics and evaluation tools will assess the program’s effectiveness, including the number of trainees who have participated in the orientation and group supervision training, immediate and delayed survey results, and trainee accuracy on BACB form completion. Displaying progress toward goal achievement maximizes impact, so a visual representation of the data should be shared within the organization. Reports should include results highlighting findings on relevance and engagement as well as accuracy in form completion.
Abstract (if available)
Abstract
Clinical supervision is a formative experience for students working toward their board certification in behavior analysis. The purpose of the study was to learn about students’ supervision experiences, gain an understanding of current practices and preparation, and determine potential barriers to successful supervision. This study used the gap analysis framework to identify knowledge, motivation, and organizational factors that influence supervision experiences for aspiring board certified behavior analysts. A qualitative methods design included semi-structured interviews and document analysis with 10 students who were currently enrolled in behavior-analytic coursework at private universities in the northeast and accruing experience hours under a qualified BCBA supervisor. Based on the findings, this study offers recommendations to close validated knowledge, motivation, and organizational gaps using the New World Kirkpatrick Model to help BCBA trainees maximize their supervisory experiences.
Linked assets
University of Southern California Dissertations and Theses
Conceptually similar
PDF
A framework for customer reputation evaluation: an innovation study
PDF
Closing the safety gap for airlines in emerging markets using the gap analysis model
PDF
Aligning digital technology to support motivation, physical activity, and sports
PDF
Reducing gaps in healthcare delivery quality on American Indian reservations and Alaska Native communities: an improvement study
PDF
Knowledge, motivation, and organizational factors that influence diversity in volunteers in animal-assisted interventions
PDF
IRB insiders: perspectives from within the institutional review board
PDF
Lesbian, gay, bisexual, transgender, and queer health curriculum in medical education: a gap analysis
PDF
Lived experiences of transgender young adults transitioning during the COVID-19 pandemic
PDF
Elevating mentees leads to organizational success
PDF
Knowledge sharing: valuing everyone at the table
PDF
Identification of barriers to mentoring and their impact on retention and advancement of underrepresented populations in the federal government
PDF
Knowledge, motivation, and organizational influences impacting senior living performance goals to reduce antipsychotics: an innovation study
PDF
Incorporating social and emotional learning in higher education: a promising practices based development of authentic leadership
PDF
Knowledge sharing among student affairs professionals at a small faith-based higher education institution
PDF
Compassion fatigue among social work students: a gap analysis
PDF
The moderating role of knowledge, motivation, and organizational influences on employee turnover: A gap analysis
PDF
Optimizing leadership and strategy to develop an expenditure-reduction plan: an improvement study
PDF
The role of organizational leaders in creating sustainable diversity, equity, and inclusion initiatives in the workplace
PDF
Lack of Latinx senior executive service members in a federal government agency
PDF
White principals implementing systems of equity: analyzing knowledge, motivation, and organizational support
Asset Metadata
Creator
Kahana, Carrie A.
(author)
Core Title
Maximizing BCBA trainee development during supervision: knowledge, motivation, and organizational influences
School
Rossier School of Education
Degree
Doctor of Education
Degree Program
Organizational Change and Leadership (On Line)
Degree Conferral Date
2022-05
Publication Date
05/08/2022
Defense Date
04/11/2022
Publisher
University of Southern California
(original),
University of Southern California. Libraries
(digital)
Tag
BCBA,clinical supervision,fieldwork,OAI-PMH Harvest,Supervision,trainee
Format
application/pdf
(imt)
Language
English
Contributor
Electronically uploaded by the author
(provenance)
Advisor
Tobey, Patricia (
committee chair
), Andres, Mary (
committee member
), Regur, Carey (
committee member
)
Creator Email
ckahana@usc.edu
Permanent Link (DOI)
https://doi.org/10.25549/usctheses-oUC111296824
Unique identifier
UC111296824
Document Type
Dissertation
Format
application/pdf (imt)
Rights
Kahana, Carrie A.
Type
texts
Source
20220509-usctheses-batch-940
(batch),
University of Southern California
(contributing entity),
University of Southern California Dissertations and Theses
(collection)
Access Conditions
The author retains rights to his/her dissertation, thesis or other graduate work according to U.S. copyright law. Electronic access is being provided by the USC Libraries in agreement with the author, as the original true and official version of the work, but does not grant the reader permission to use the work if the desired use is covered by copyright. It is the author, as rights holder, who must provide use permission if such use is covered by copyright. The original signature page accompanying the original submission of the work to the USC Libraries is retained by the USC Libraries and a copy of it may be obtained by authorized requesters contacting the repository e-mail address given.
Repository Name
University of Southern California Digital Library
Repository Location
USC Digital Library, University of Southern California, University Park Campus MC 2810, 3434 South Grand Avenue, 2nd Floor, Los Angeles, California 90089-2810, USA
Repository Email
cisadmin@lib.usc.edu
Tags
BCBA
clinical supervision
fieldwork