Repository navigation
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|
Repository files navigation
README
This repository contains supplementary materials for:
“Evolutionary dynamics of collective behavior across the tree of ray-finned fishes”
Jay Love, Amlan Nayak, Abigail G. Grassick, Ling-Wei Kong, Ella Henry, Michael A. Gil, Ashkaan K. Fahimipour, Matt Pennell, Andrew M. Hein
Accepted for publication in:
Proceedings of the National Academy of Sciences of the United States of America
doi:10.64898/2026.04.06.716557
Please contact Jay Love with questions, issues, or concerns.
This README document serves as a guide to the materials. Please also refer to SI section 7.
files
_____
-write_work_dir.py
- run to write a file in the 'supplement' directory containing the full path to the directory, for use in other scripts.
-install_dependencies_f.R
- run to install all R dependencies needed for the full data generation and analysis pipeline. Also downloads the phylogenetic tree and compiles large files from components.
- ./data_generation_pipeline/
- directory contains scripts to generate data used in the paper.
- NOTE: Running the full data generation pipeline could take several days on a personal computer, and a NVDIA GPU is required unless the code is edited to suit. While we have done our best to ensure the code runs on all systems, the particulars of local enviornments may cause runs to fail due to, for example, memory limits. Most readers will want to skip to "results_f.R".
- In order of operation:
- README_text.md: instructions for establishing a python environment for the text classifier used in trainloop_f.py
- trainloop_f.py: trains a series of text classifier models on different train/test splits and with different levels of synthetic training data fortification/augmentation
- prep_f.R: processes data, handles species name matching between FishBase and Rabosky et al. nomenclature
- create_f.R: fits threshold and hisse models
- results_f.R: conducts main analyses used in paper. Most readers will want to start here.
- ./image_pipeline/
- README_image.md: instructions for establishing a python environment for the image ML custom pipeline used in the annotate.py custom GUI program.
- annotate.py: custom-built GUI for custom ML pipeline to process images. Runs from the command line following instructions at the head of the script.
- comp_data.csv
- contains the data used in the primary analyses in Love et al. Data dictionary follows:
- SpecCode: numeric; species code used in FishBase
- phyloname: string; binomial used in the phylogenetic tree (updated from Rabosky et al. 2018 - see ./data_generation_pipeline/prep_f.R)
- Comments: string; Contents of field from Biology table in FishBase species record
- adultschool: numeric; probability of classification as "collective" by the text classifier model
- adultsolitary: numeric; probability of classification as "solitary" by the text classifier model
- juv: string, interpret as factor; manual classifications as solitary or collective (phrased as "schooling" in the data here) of non-adults
- fromset: string; was this taxon included in the groundtruth set (gt), predicted set (pred), or as additional taxa included as part of the dimensions dataset (xtra)?
- pschool: numeric: proportion of time spent in the collective state (if known)
- pschoolref: string; reference for 'pschool' field
- species: string; duplicate of 'phyloname' for matching downstream
- liabbert: numeric; liability returned from the threshold model fit to data in 'adultschool' and 'adultsolitary', phrased "c" for "collectiveness" in the text of Love et al.
- troph: numeric; trophic level estimate
- len; numeric: body length for ancillary analysis
- lentype; string; TL (total length) or SL (standard length) for field len
- adultbert; string, interpret as factor; binary trait classification, taken as higher of 'adultschool' and 'adultsolitary'
- groundtruth_reference.csv
- contains the groundtruthed records, with citations and details of review and QC processes.
- fortification_text.csv
- contains the synthetic texts used to augment the training data for the text classifier model. These texts are also hardcoded into ./data_generation_pipeline/trainloop_f.py
- classifier_sensitivity_f.R
- contains a script to interactively compile and evaluate reports of text classifier model fits
- ./classifier_sensitivity/
- a directory containing .csv file outputs of text classifier model fits, which vary in the proportion of ground-truth data used in training and the degree of training data fortification used. These files are compiled into a more readable format using classifier_sensitivity.R, above.
- pCollective_set.csv
- contains the records on species (column 'sciname') for which data were available on the proportion of time spent in a collective state ('pcollective'). Also contains stable links to images from public databases ('img_X'), and reference for the 'pcollective' value ('ref').
- ./dims/
- a directory containing the data for the dimensions of collective behavior analysis, with filenames indicating species name and public database accession number.
- attributions.csv
- a table of the inaturalist image accession numbers
- ./etc/
-a directory containing assorted materials for reproducing results in the scripts, primarily results_f.R