Presented by
Presented by
The toolkit framework and content is based on findings from two research activities:
The Institute for Community Inclusion partnered with The Arc of the United States to conduct both studies.
Learn more about the Delphi process.
Learn more about the case study research.
A Delphi process is a research technique used to obtain consensus from a group of experts on a particular topic. The following explains a) the recruitment of Delphi panelists, b) who the panelists were, c) data collection techniques, and d) data analysis techniques.
The research team implemented a variety of recruitment methods to establish a group of experts in the field of organizational transformation. This began by contacting individuals within their professional networks with experience in this area. Project advisory board members were also asked for suggestions.
The team also identified a number of other published academics in the subject of organizational transformation, based on their own knowledge of the field and on available literature.
As a result of the recruitment process, a final list was comprised of 44 candidates. Of the 44, 36 individuals responded to both rounds of surveys (described below).
The majority of the Delphi panel was female, Caucasian, between the ages of 51 and 70, and reported having earned a master’s or doctoral degree. Their work was geographically distributed across 44 states, and half of the panelists (50%) were or previously had been in leadership or administration roles within provider organizations.
The panelists displayed a vast amount of knowledge and experience in the organizational transformation process. Most had been working in the field for over 20 years, and had experience in either participating in or leading organizational transformation efforts in their past or present positions (68%).
Twenty-six panelists (77%) reported being part of an organizational change process, with 20 panelists (59%) having served in a leadership role during a transformation process.
Data collection began by emailing a link to a SurveyGizmo survey containing the original six T-TAP characteristics identified in previous ICI research.
In this survey, panelists were asked to review the six characteristics and add any characteristics they identified as missing (Round 1). This resulted in the Delphi panel identifying ten characteristics necessary for successful organizational transformation.
After this analysis, panelists were emailed a link to a second SurveyGizmo survey containing these ten characteristics, in which they were asked to rank them according to their importance (Round 2).
Data was comprised of the panelists’ recommendations about the elements necessary for organizational change. The researchers aggregated responses and created a list of codes based on common themes.
The researchers then used the coding list to categorize the dataset into recommended characteristics. These characteristics would contain language and definitions drawn from text provided by the panelists.
This list of ten characteristics was sent back to the panelists via email for verification of researcher interpretation. Panelists were asked if the characteristics and their definitions were both accurate and appropriate.
In the second round, the panelists were then asked to rank the elements according to their relevance to organizational transformation.
Learn more about the Delphi process findings. [LINK TO DELPHI BRIEF IN SEPARATE DOC]
The following explains
Recruitment materials were widely distributed through national listservs and email distribution lists. Twenty-eight organizations volunteered to participate in the study.
To narrow down participants, the study team developed the following criteria for inclusion:
After applying these inclusion criteria, eight organizations were selected to participate in a telephone screening call. The telephone screening was conducted with key leadership from each organization, and lasted about an hour.
Screening interviews served two purposes. The first was to create an organizational sketch for each case study site by obtaining detailed demographic information. The second was to obtain an overview of the transformation process, including the organization’s strategic planning, stakeholder involvement, funding and communication strategies, and partnerships that supported the change.
Based on the information gathered during the screening interviews, four finalists were chosen. Geographic location, numbers served, and state population were taken into consideration when choosing finalists in an effort to obtain a representative sample.
Learn more about the findings of the screening interviews. [LINK TO SCREENING BRIEF IN SEPARATE DOC]
Overview of the four finalists
Prior to the in-person site visit, key staff at each organization were identified to support the creation of an agenda and identify the most appropriate participants. On-site interviews were conducted individually and in small groups over the course of 2-3 days.
Open-ended questions varied slightly based on respondent group, and were focused on gaining information relevant to each of the ten elements resulting from our Delphi panel. [LINK TO DELPHI BRIEF IN SEPARATE DOC] Agency leadership (e.g., administrators and board members), front line staff (e.g., direct support providers and supervisors), external stakeholders (e.g., state agencies, consultants, employers), individuals, and family members were all included.
Interviews were recorded, transcribed, and uploaded to Atlast.ti software for coding. Project staff read through each transcript and developed a list of codes, or general themes, based on initial reactions. Codes were discussed during regular team meetings, and a final list was agreed upon.
Once the code list was finalized, team members read and coded all transcripts using this list. Once coding was complete, quotes for each code were bundled into one document. From those lists of quotes associated with each code, themes were identified using the structure of the ten elements from the Delphi process.