The volume of subcortical structures represents a reliable, quantitative, and objective phenotype that captures genetic effects, environmental effects such as trauma, and disease effects such as posttraumatic stress disorder (PTSD). Trauma and PTSD represent potent exposures that may interact with genetic markers to influence brain structure and function. Genetic variants, associated with subcortical volumes in two large normative discovery samples, were used to compute polygenic scores (PGS) for the volume of seven subcortical structures. These were applied to a target sample enriched for childhood trauma and PTSD. Subcortical volume PGS from the discovery sample were strongly associated in our trauma/PTSD enriched sample (n = 7580) with respective subcortical volumes of the hippocampus (p = 1.10 × 10−20), thalamus (p = 7.46 × 10−10), caudate (p = 1.97 × 10−18), putamen (p = 1.7 × 10−12), and nucleus accumbens (p = 1.99 × 10−7). We found a significant association between the hippocampal volume PGS and hippocampal volume in control subjects from our sample, but was absent in individuals with PTSD (GxE; (beta = −0.10, p = 0.027)). This significant GxE (PGS × PTSD) relationship persisted (p < 1 × 10−19) in four out of five threshold peaks (0.024, 0.133, 0.487, 0.730, and 0.889) used to calculate hippocampal volume PGSs. We detected similar GxE (G × ChildTrauma) relationships in the amygdala for exposure to childhood trauma (rs4702973; p = 2.16 × 10−7) or PTSD (rs10861272; p = 1.78 × 10−6) in the CHST11 gene. The hippocampus and amygdala are pivotal brain structures in mediating PTSD symptomatology. Trauma exposure and PTSD modulate the effect of polygenic markers on hippocampal volume (GxE) and the amygdala volume PGS is associated with PTSD risk, which supports the role of amygdala volume as a risk factor for PTSD.