Data-driven Team Development: Creating heroes with ones and zeros

The research is based on findings from studies encompassing a literature review on media-rich externalized knowledge representations, a large-scale cyber defence exercise, and a controlled virtual team experiment. The Baltic Cyber Shield exercise provided an opportunity to observe team dynamics, strategy development, and performance measurement in a realistic cyber defence scenario. Collaborative decision-making in defensive cyber operations was at the time a little-studied area, and the exercise offered unique insights into how ad-hoc teams operate under pressure in a complex environment. The second experiment, conducted at Hartnell College, enabled a detailed comparison between self-assessment and observer ratings in virtual teams, highlighting the challenges of evaluating team effectiveness. The virtual team setting at this experiment allowed for collection of rich data on team interactions and performance, enabling another study on the feasibility of automation for team performance assessment in controlled environments.

Collected data included surveys, observer reports, system logs, and automated performance metrics. These data were analysed using statistical methods and regression analysis to identify relationships between team behaviours, decision-making processes, and outcomes. The studies reveal that data-driven analysis not only supports experience sharing but can also be used to assist in performance assessment, offering a complementary approach to traditional self-ratings and observation-based assessments.

Key findings indicate that data-driven approaches can indeed accelerate team development, e.g. by strengthening the feedback loop between team members and enhancing their ability to learn from each other. Integrating automated performance metrics with observer and self-assessment data enables a more comprehensive analysis and understanding of team performance and effectiveness. The research highlights that diverse data sources can reveal complementary strengths and limitations, enabling more accurate identification of areas for improvement and supporting targeted interventions in team development.