OVERVIEW OF THE COTAS VALIDATION
The COTAS validation focused on three components: (1) the predictive measures for violence and the thresholds used to distinguish levels, (2) the programming and software implementation, and (3) the ability of COTAS to provide accurate, understandable, meaningful, and constructive information to end-users. The validation of the predictive measures for violence and the establishment of thresholds focused on two elements of predictive accuracy: (1) the appropriateness of the modeling procedure and (2) the discrimination of the predictive model. The appropriateness of the modeling procedure refers to the correct selection and application of the statistical model. For example, are all of the mathematical assumptions of the data satisfied? Discrimination refers to the model’s ability to distinguish between low-risk and high-risk events and, therefore, the ability to set significant threshold variations. For example, are inmates with high risk scores involved in a higher proportion of violent events relative to inmates with low risk scores?
The validation of the software implementation and aggregation of data was conducted by ELENC, Inc, a private consulting and software design firm. The software validation procedures were divided into five key evaluation tasks:
- Software design evaluation (e.g., how well is the software designed and coded?
- Software Implementation (e.g., how well is the application implemented?)
- User Documentation (e.g., are the materials accurate, sufficient, and up-to-date?)
- Software Transferability (e.g., can the software be transferred to and used by another agency in a cost effective manner?)
- Software Extensibility (e.g., can new software features and functions be added for a reasonable cost?)
The software validation methodology and results are presented in Chapter 6.
An assessment of COTAS users’ experience was conducted through a web-based user feedback survey, which examined users’ experiences with and knowledge of the system and provides suggested improvements to COTAS. Surveys were administered to correctional administrators at all DOC facilities in the state.
Structure of the report
Chapter 2 of this validation report presents an overview of COTAS including a detailed description of the functions and the user interface. Chapter 3 provides an overview of the validation including a description of the methodology, an analysis of the model’s ability to predict inmates’ involvement in violent events, and the results. Chapter 4 examines users’ experiences with COTAS and presents the findings of the survey of COTAS users. Chapter 5 summarizes the COTAS validation and presents recommendations that may improve the accuracy and usability of COTAS. Chapter 6 presents ELENC, Inc.’s software validation report which includes an examination of the ETL procedure for the data warehouse and the software design of COTAS.
Contacts
Principal Investigator: Thomas Blomberg, Dean and Sheldon L. Messinger Professor of Criminology
Researchers: Jim Clark, William Bales, Karen Mann
Graduate Student: Leslie Hill
Funding Agency: Florida Department of Corrections
Funding Amount: $140,941
Date: 2011