In a study published in Neuron, the researchers from Dr. XU Xiaohong’s Lab at the Institute of Neuroscience, Center for Excellence in Brain Science and Intelligence Technology of the Chinese Academy of Sciences, described a novel neural circuit for detecting male pheromone cues pertaining to inter-male aggression. 

Innate social behaviors are crucial for the propagation of a species. Yet, before launching any consummatory actions, animals need to first integrate sensory information to determine the sex of the interacting conspecifics. For example, males mate towards females but attack other males.  

Previous studies have shown that when the olfactory vomeronasal organ (VNO) was disabled by knocking out the gene encoding transient receptor potential cation channel, subfamily C, member 2 (trpc2), male mice showed diminished inter-male aggression and increased mating behaviors towards both male and female intruders. This indicates that a mouse’s ability to discern the sex of a conspecific critically depends on olfactory information from the VNO.  

Recently, a number of studies have established that the hypothalamus plays an important role in the regulating innate social behaviors such as mating and aggression. But, how these hypothalamic neurons integrated sex information from the VNO to promote appropriate behavioral outputs remains largely unknown. 

The researchers in this study, using calcium transients recording in behaving mice, found that a group of dopamine transporter (DAT) positive neurons in the ventral premammillary nucleus (PMv^DAT) responded strongly when a male mouse sniffed male urine, but not urine from castrated male or female mice, or newborn mice, or rats.  

When they inhibited the PMv^DAT neurons by chemo-genetics, the males lost their ability to recognize male urine and at the same time showed reduced attack behaviors towards a male intruder. Conversely, activating PMv^DAT neurons increased the aggressive behaviors towards male intruders.  

Then, the researchers used trpc2 knockout mice to specifically destroy the VNO signaling. They found that the response of PMv^DAT neurons to male urine in knockout males decreased significantly.  

Furthermore, using virus tracing, calcium recording and chemo-genetics, the researchers found that male-relevant olfactory information was relayed via the bed nucleus of ventral stria terminalis (v-BNST) from the VNO to PMv^DAT neurons.  

These results demonstrated a critical role for PMv^DAT neurons in encoding male conspecific information relevant to inter-male aggression.  

This study provide crucial insights into the neural circuit mechanism underlying sex specific olfactory information processing and regulation of sexually dimorphic social behaviors.