2015 Winner — Social Security Administration, Office of Disability Adjudication and Review, Office of Appellate Operations (ODAR/OAO)
One of the largest administrative judicial systems in the world, the Social Security Administration’s ODAR/OAO is responsible for the final two levels of the administrative appeals process for SSA disability claims. The Office faced a growing backlog of cases, and the massive size of their workload demanded that they systematically and strategically approach the challenge of improving their processes.
To reduce their case backlog, ODAR/OAO needed to both improve accuracy (thus reducing the number of remanded, or “re-tried,” cases) and increase case processing speed. Sampling a broad cross-section of Administrative Law Judges (ALJs) responsible for reviewing the cases, the organization benchmarked their current skill sets and determined program goals against 84 metrics by which ALJs can be evaluated. Curriculum designers considered best practices in adult education and stressed the critical-thinking component of the learning process. The final training product was a 2-day train-the-trainer program and a 1-week advanced training program for the ALJs.
“We understood that to achieve a meaningful and lasting improvement in the quality of our services,” said Glenn Sklar, Deputy Commissioner, ODAR, “we must use a data-driven approach to get at the heart of the issues we face and produce meaningful and measurable results.”
For ODAR/OAO, the results of this initiative were eye opening. After the training, the Office saw a 5% or greater improvement in 35 of the 84 performance metrics of ALJs. They also saw a 2% - 4.99% improvement in 14 of the 84 metrics. Statistically speaking, this training had a significant impact on ALJ performance.
Internal response was overwhelmingly positive: judges found the experience “invigorating,” and many said the training got to the heart of issues, pushing them to think in entirely new ways. Not only did the training improve the work process, it inspired participants to take what they’d learned and share it with other judges.
The success of this project validates the organization’s efforts to use data to drive decisions about policy and training for all employees. ODAR/OAO developed an effective training that achieved statistically significant results and also created enthusiasm among participants—increasing the probability that information will be retained, applied, and shared among colleagues.
“Being recognized for our training efforts to improve the disability appeals process is validation of ODAR’s hard work over the last several years,” said Patricia Jonas, Executive Director, OAO. “But the real reward comes from knowing that it is helping us achieve our mission to provide people who seek disability benefits with a timely, fair, and accurate decision, while complying with all laws, regulations, and policies.”
2014 Winner — Federal Emergency Management Agency (FEMA) Employee Development Division (EDD)
The Federal Emergency Management Agency (FEMA) Employee Development Division (EDD) was recently tasked with changing the agency’s culture of leadership across all levels of responsibility. A primary goal of FEMA’s workforce enhancement initiative, this change entailed improving areas such as leadership trust, accountability, and performance. Individuals across all levels needed to develop competency in leader proficiencies and learn management skills required for success, and supervisors needed to be educated about their role in developing employees and prioritizing learning needs.
Faced with budget constraints and the need to provide training for agency employees nationwide, the EDD knew that its activities, workshops, and programs should be transportable, scalable, and inclusive. This goal led to the design and successful implementation of an Individual and Organizational Progressive Leader Development Program designed to build proficiency in leader competencies.
The wide-scale, innovative model uses multiple instructional strategies and keeps the needs of adult learners at its center. All training elements, which meet the established standards, are designed with a focus on mission-critical leader competencies. A significant percentage of training comprises non-classroom activities that benefit individual and organizational growth. These include workshops and programs for employees that range from GS-specific training to an internal mentoring program; a book club; webinars; and a workshop series.
Key learning concepts are taught across the organization—to team members, team leads, supervisors, managers, and executives — and are also used in team - and organization building activities and general workshops. This has enabled FEMA to begin developing common learning tools using terms and concepts recognizable to all.
A review of the first (FY13) cohort revealed measurable outcomes which include 20% of program participants being promoted into supervisory positions or receiving grade level increases, or transitioning from “on-call response employees” to permanent, full-time agency employees. Additional measurable outcomes included an increase from 50% to 92% in supervisory-required training compliance within six months of program participation. The EDD, serving as an in-house resource for leadership development, continues to provide training-related individual and developmental services to employees who have completed the program.
2013 Winner — Department of Housing and Urban Development, Office of Field Policy and Management
The Department of Housing and Urban Development’s (HUD) Office of Field Policy and Management seeks to ensure that its performance and use of resources result in detailed and accurate performance measures. In response to congressional inquiry, HUD sought to integrate an intervention to improve its track record in reporting how the agency addresses its organizational initiatives.
Previous attempts to implement reporting methods had been unsuccessful because they were not performance-oriented, required extensive staff time, and failed to provide concise, objective data results. It was essential to HUD’s success that it improve the ability of field members to track their projects, demonstrate alignment to HUD strategic goals, and report out progress updates.
The winning intervention was a Training and Orientation Program (TOP) designed to reinforce the use of a cloud application for tracking and reporting project performance. With field and headquarters staff consisting of hundreds of employees in 80 field offices throughout the U.S., and with oversight by 10 regional headquarters, implementation presented unique challenges.
Users at all locations received in-class, hands-on training in the new cloud application and then completed online testing and certification. The app itself is designed to include embedded help and guidance, as well as an audiovisual “how-to” program. TOP was implemented from October 2011 to July 2012, and it continues today with various enhancements based on continuous employee feedback.
This classroom/online training combination, paired with ongoing user support provided by the app itself, has been enormously successful. The cloud system is now used by 100% of the staff, with 25-30% logging in daily to perform project-related work. Additional benefits of the program are the acceptance of a common method for project planning and reporting, and the availability of a common real-time view of progress at various leadership levels. Numerous field recommendations have resulted in improved reporting to Congress and others. As a benefit extending beyond the reporting aspects of the intervention, this program has presented an opportunity to introduce performance and process philosophies to a key set of federal executives.
2012 Winner — Naval Aviation Test and Evaluation University, Naval Air Systems Command
The Naval Aviation Test and Evaluation University (NATEU) was developed by the Naval Air Systems Command to provide training to Department of Navy flight test personnel and to share this training with other Department of Defense entities and industry partners. Unlike other areas of postgraduate study, comprehensive test and evaluation engineering training is not available through any traditional academic institutions. The University was developed in response to a challenge from the Chief of Naval Operations to “improve training and development opportunities for our civilian workforce, recognizing that their individual growth is an institutional gain.”
The NATEU results were impressive. They aligned training with a career path progression; created comprehensive and specialty training; partnered with industry for team training; improved testing and evaluation execution with Naval Air Systems Command; and increased organizational efficiencies. In just three years, they have also created over 100 core and specialty training courses which revitalized the Naval Air Systems Command workforce by fully developing the organization’s pool of qualified test and evaluation practitioners. Much of this was done through the use of cohort training, building a more cohesive and collaborative workforce.
Update on NATEU since the award: At the time of the award entry, NATEU had trained 2,000 students; now they have trained 4,000 students. With the training framework and infrastructure, they have been able to create an accelerated reach into the organization as well as within the broader Department of Defense.
What the Deming Award has meant to their agency: “The Deming Award served as our ‘Underwriters’ Laboratory’ stamp of approval. It validated our training efforts on a grander scale outside our own Navy perspective and enabled us to gain recognition within the DoD."
—Stephen E. Cricchi
Director, Integrated Systems Evaluation,
Experimentation and Test Department
Naval Air Systems Command