Tests for introgression
The most supported ADMIXTURE result parsed six populations (K = 6; Fig. 3a): two within each marten species and two putative hybrids, identified as independent populations (KUI, MTX). Although K6 exhibited the greatest Δcv (Supplemental Information 9), we also examined alternative values of K (Lawson et al. 2018; Supplemental Information 10) to thoroughly characterize genetic structure in North AmericanMartes (Janes et al. 2017). The K2 model splits M. caurinafrom M. americana , while K3 through K5 models identify additional intraspecific structure. Insular and continental M. caurinapopulations were separated under K4 and K6 models, while K7 distinguished each M. caurina sample as an independent population (Supplementary Information 10-11). Under all values of K, only two individuals were identified as admixed: KUI and MTX. NegativeF3 -statistics (Table 2) also consistently identified these samples as hybrids between M. americana and M. caurina , while all other F3 results were positive or insignificant (Supplemental Information 12-13). Consistent with geography and a K3 ADMIXTURE model, F3 -statistics, among populations identified by K6, found the KUI hybrid to be the result of mixture between the insular populations of each species, while the continental hybrid appears to be a combination of the continental populations of each species (Table 2). Under K2 through K4 models, hybrid samples contain near 50% genetic proportions corresponding to each species, suggesting that these individuals may represent F1 hybrids or early generational backcrosses (Supplemental Information 11). Consistent with these results, hybrid-class simulations most support the KUI hybrid (empirical admixture proportions: 55% M. americana , 45% M. caurina ) as an F1 or 48-54% assignment to each species. Empirical admixture proportions for the MTX hybrid (60% M. americana , 40% M. caurina ) fall intermediate to simulated admixture proportions for an F1 hybrid and a single generation backcrossed with M. americana(73-76% assignment to the backcrossed species, 22-26% assignment to the other parental species). Results were identical for simulations on all M. americana and M. caurina and for insular and continental populations of each species, when run separately, indicating that both hybrids represent early generational stage crosses, with MTX potentially the result of a complex backcrossing history between both species. R simulations based on Lavretsky et al. (2016) demonstrate the loss of a signature of introgression after four generations of backcrossing into either species (Supplemental Information 14).
Consistent with those backcross estimates, Treemix phylogeny estimations rooted on M. zibellina placed both hybrids as separate sister lineages to the M. americana clade and identified the first most likely gene flow event between MTX and an M. caurina ancestor, followed by a migration event between KUI and M. caurina(Supplemental Information 15). Interestingly, the third most likely migration event was estimated between the outgroup M. zibellinaand the KUI hybrid.
F4 statistics show the directionality and intensity of backcrossing is similar across the two hybrids (Supplemental Information 16-17). KUI shares more genetic overlap with insular M. caurinacompared to the continental clade and MTX has similar proportions of insular and continental M. caurina alleles (Supplemental Information 16-17). For tests across all individuals, AdmixtureGraph identified KUI as a mix between CHI and POW (minimal error = 0) and MTX as a cross between ADM and the ancestor of continental M. americana (Supplemental Information 18-20). Interestingly, when considering only the populations identified by ADMIXTURE (e.g., pooled continental and insular populations of each species), AdmixtureGraph identified both hybrids as a mix between the continental populations of each species. However, notably there were multiple clustered minimal-error estimates (Supplemental Information 20) that we anticipate will be resolved with increased sample sizes. MixMapper found that the timing of admixture did not differ from the present (0) for either hybrid (Supplemental Information 21).