While the FDP application certainly requires staff dedicated to maintaining it, the exact skillset of that staff depends on your needs and desires for the application. Here we break down examples of the various responsibilities you might want staff to perform and the different skill sets needed to execute on those responsibilities.
While one person may have overlapping skills, there is no expectation that any one person would have all of the skill sets described in this document. Likewise, these are not necessarily four different job descriptions.
We recommend starting with staff that can at least perform the Data Collection and Data Processing functions. Responsibilities are listed in order of approximate level of importance, beginning with those we believe are necessary to maintain the database – Data Collection and Data Processing - followed by those that would allow for additional analysis and customization if you choose.
Responsibilities:
- Data Collection & Entry
- Data Processing
- IT Responsibilities
- Data Analysis and Reporting
- Application Development
Data Collection & Entry 
These skills are essential for maintaining the FDP application.
Responsibilities
- Research sources of police misconduct data
- Draft and execute the organization’s Data Collection Plan
- Request and collect data on a regular weekly, monthly, or annual bases from online sources, city agencies, case management databases, and through Freedom of Information Requests
- Collect information and records from attorneys and other personnel regarding police misconduct.
- Process confidential attorney-client privileged data and information
- Develop data entry manual with guidelines for how to enter data into database
- Enter all data into database, following strict protocols to ensure data integrity and confidentiality
- Maintain quality assurance of collected data
- Manage user permissions to the database
- Work with consultants on any database system errors or upgrades
- Function as point of contact for all user issues or troubleshooting
Example Tasks
- Make Freedom of Information Requests to collect the rosters for each law enforcement agency, specifying what fields to include, and follow up when the agency does not send all the requested information
- Gather copies of all the civil lawsuits against law enforcement agencies posted on a court’s online portal and enter into database
- Decide on standards and protocols for how each source of information should be entered in the database, such as a civil rights lawsuit or a civilian complaints
Required Skills
- Attention to detail, experience in online research
- Ability to evaluate datasets and their completeness
- Knowledge of relational databases and ability to understand relational schemas
- Bonus – familiarity with criminal defense function and police misconduct data
Data Processing 
These skills are essential for maintaining the FDP application. They are necessary for any bulk data processing.
Responsibilities
- Extract and parse text from unstructured (printed, scanned, or electronic) data
- Write and execute scripts to process unstructured data into a format that matches the database model
- Bulk import data into the database, carefully following guidelines to serialize and map the data
- Utilize advanced data processing techniques for deduplication and records linkage, like fuzzy and probabilistic matching
- Advise on the scope and limits of the data
Example Tasks
- Given 10 years of police rosters in .csv files, with no single unique officer identifier, create a list of unique officers with their agency start and end date, accounting for misspellings, name changes, officers with the same name etc
- Write a script to be used bi-annually when a new roster comes out to match officers to those currently in the database and identify those that are new, promoted, or retired
- Given a monthly set of 50 pdfs of civil lawsuits, extract the text of the files, locate a unique identifier, cross compare with lawsuits in the database, update their status, and add the new lawsuits to the database
- Identify the 5 most common pdf document data types and write scripts to routinely extract their data and digest into the database
Required Skills
- Experience in data manipulation, normalization, and extraction
- Ability to write and execute scripts in languages for manipulating data (ex Python or R)
- Knowledge about data schemas and relational data models
- Experience culling heterogenous data from a variety of sources
- Experience with data scrubbing tools
- Experience working with confidential, sensitive, and large, messy unstructured data
- Ability to recognize (OCR) and parse text from document data
- Helpful skills include familiarity with regular expressions and packages for manipulating strings (dplyr, stringr, pandas, etc.)
IT Responsibilities
Existing IT staff may play an important role in the implementation of the application at your organization. The following worksheet lists the typical responsibilities for setup and maintenance of the FDP App. software. Typical assignments are offered below as an initial default. These roles should be reviewed carefully and revised at the beginning of the project. If your organization has little or no dedicated IT staff, it is still possible to set up the application under special cases, subject to funding and availability.
Installation
- Azure portal billing and subscription setup (client)
- Grant FDP 'contributor' role access to resource group (client)
- Resource performance tier scaling (both FDP and client)
- Services purchasing (both FDP client)
- Azure services backups and redundancy settings (FDP or client)
- If required: Implement backup procedures and protocols beyond default maximums provided by Azure services (both client and FDP)
- If custom domain: domain and DNS Server configuration (client)
- If custom domain: HTTPS certificate (client)
- App Service runtime and web proxy setup (FDP)
- FDP App codebase installation and configuration (FDP)
- PostgreSQL server and database setup (FDP)
- File storage account and container setup (FDP)
- If single-sign-on: Active directory API Client setup (client)
- If single-sign-on: Active directory authentication integration (FDP)
- Security checklist run through (both FDP and client)
- If required: Implement security monitoring and auditing protocols (client)
- If required: implement additional security systems such as firewalls (client)
Ongoing service maintenance and administration
- Grant user access and authorization in system (client)
- Carry out security monitoring and auditing protocols (client)
- Restore system from data backups in disaster recovery scenario (FDP or client)
- Monitor system performance and scale infrastructure to fit demand (both client and FDP)
- Carry out data backups procedures and protocols beyond those provided by Azure services (client)
- Upgrade application code to security releases and new versions (FDP)
- As needed: Maintain application code, fixing bugs, adding new features (FDP)
- Upgrade operating system, and runtime stack (all three Microsoft Azure App Service, FDP, and client)
- If required: maintain additional security systems such as firewalls (client)
Data Analysis and Reporting 
These skills not necessary. They are helpful only if you want to move beyond looking at individual officers’ misconduct history in your data and analyze misconduct trends and patterns.
Responsibilities
- Write commands to extract data from a PostgreSQL database
- Explore and identify trends of law enforcement misconduct in the data
- Answer questions about police misconduct patterns from attorneys, journalists, public officials, etc
- Evaluate what research questions are and aren’t possible to answer given the scope of data
Example Tasks
- Pull and analyze data from the database for a report on whether use of force incidents changed in precincts when a lieutenant was promoted
- Find the command with the most incidents of force per officer
- Perform a social network analysis on lawsuit data to find officers frequently sued together
- Identify if there are differences in discipline for the same misconduct for officers of different ranks
Required Skills
- Familiarity with basic SQL commands to pull data from a PostgreSQL database
- Ability to understand complex relational data models
- Experience and sensitivity to working with confidential and protected information
- Ability to analyze both quantitative and qualitative data (data processing skills listed above apply)
- Knowledge of the biases and limitations of the data
Application Development 
These skills not necessary. They are helpful only needed if you want someone to make any changes to the application itself by adding additional functionalities or features.
Responsibilities
- Develop features and enhancements for an open-source Django based information system
- Work with stakeholders to elicit requirements for features and changes and coordinate version updates
- Follow provided guidelines for customizations provided by consultant
- Write modular code that interfaces with existing code base of approximately thirty-four thousand lines of code
Example Tasks
- Create a new search interface for the application that allows users to search document text
- Customize the officer profile to show additional fields
- Integrate the system with a case management system
- Develop reporting tool to generate exports of data for analysis and metrics gathering
Required Skills
- Demonstrated professional experience working with the Python programming language and the Django web development framework
- Experience with PostgreSQL preferred, or other relational database
- Experience in writing unit and integration tests
- Deep knowledge of HTML/CSS
- Familiarity with Git or other version control system
- Experience with free and open-source software like GNU/Linux and command-line tools
â—„ How to Track Police Misconduct
