High‐quality habitats are predicted to have higher densities of competitors, smaller territory sizes, and increased defensive costs for territory holders. However, defence is typically considered to be directed predominantly towards conspecifics, with heterospecific intrusions generally thought to contribute less to defensive costs. Moreover, individuals may benefit from the collective defence of resources in areas with increased densities of territory holders. Together, these factors could affect the costs of defence in low‐ versus high‐quality habitats. In this study, we investigated how habitat quality and conspecific density influence the territorial defence of algal‐farming yellowtail damselfish ( Stegastes arcifrons ) in the Galapagos. We quantified territory characteristics, performed behavioural observations, and conducted an in situ experiment to test whether damselfish in higher‐quality, conspecific damselfish‐dense habitats (characterised by increased algal turf coverage) differ in their competitive intrusion frequencies and defensive costs of chasing compared to those in lower‐quality areas. We found that damselfish in poorer quality habitats with fewer conspecific neighbours had larger territories compared to those in higher‐quality, more damselfish‐dense areas. Despite this, both high‐ and low‐quality territory holders experienced equivalent numbers of competitive intrusions. However, the type of competitor varied: territory holders in high‐quality areas primarily defended against conspecifics, while those in lower‐quality areas faced more frequent heterospecific intrusions. We also found no evidence that damselfish in high‐quality habitats gained any collective benefits of defence by having increased densities of conspecific damselfish nearby. Thus, while higher conspecific density in higher‐quality habitats might alter the risk from certain intruders, it does not appear to confer direct collective defence benefits in terms of reducing defence effort once intruded. Instead, territory holders appear to adapt their defence to balance the defensive pressures posed by conspecifics and heterospecifics, maintaining similar overall defensive costs across differing habitat qualities.