Barton et al. 1995 examine the correlation of size of sensory structures in the brain with each other and with ecological factors of activity schedules and diet within primates, bats and insectivores. Controlling for brain size and phylogenetic relatedness, they show several correlations of residual contrasts between visual and olfactory brain structures that are in several ways consistent with ecological lifestyles of the species in question. For example, in primates there is an overall negative correlation between olfactory and visual structures, where diurnal species have larger visual cortices and nocturnal species larger olfactory structures. Additionally, diurnal frugivore-omnivores have larger visual cortices than diurnal foliovores. In bats, there is a positive relationship between olfactory and visual structure size, though it is suggested that these two characteristics are likely involved in a trade-off with echolocation abilities. Insectivore analyses typically had low sample sizes, though trends suggest a trade off between fossorial abilities and visual structure size. This study is well done, especially considering that it was published ten years ago. It sets the stage for genetic-based and functionally-based studies of these “trade offs” within particular groups (i.e. trichromatic vision in primates).
Gilad et al. provide a comparative study of the proportions of olfactory receptor pseudogenes across 19 primate species, finding that routine trichromatic primates have a higher proportion of pseudogenes than dichromatic (or allelic trichromatic) primates, suggesting decreased reliance on the olfactory sense relative to vision in trichromatic primates. This high proportion of pseudogenes association was also consistent for the howler monkey, the only new-world monkey to acquire routine trichromatic vision, an independent event from the evolution of trichromacy in old-world primates.
So, we see a decrease in utility of OR genes coincident with the evolution of trichromatic vision in primates, although Gilad et al. cannot show that this is due to a direct relationship between olfaction and vision. Here’s an idea—maybe can we test the relationship between olfactory and visual systems in other ways, looking at utility via the signature of neural connectivity across individuals in a population. Many (most? all?) new world monkeys have “allelic trichromacy” where the optic gene on the X-chromosome is polymorphic for sensitivity to medium and long wavelengths; as a result, female heterozygous individuals are trichromatic and female homozygous individuals and males are not. Assuming there is some advantage to trichromacy, and if the neural sensitivity in sensory systems are labile within individuals, maybe we could expect to see increased neural sensitivities in the visual brain regions with a corresponding decrease in olfactory neural sensitivity/connectivity for female heterozygous individuals compared to other dichromatic individuals in a population. ??? Another offhand comment--one concept I’m not completely clear on is “constraint”. It is easy to understand that brain size is constrained by the size of the skull cavity, or energetic expenses for development and maintenance of particular organs are constrained by the proportion of energy allocated to others, but I feel that the term is sometimes used loosely, or at least the constraining factors are less obvious. In this paper, the author suggests near the end of the discussion that while some OR genes are accumulating coding region disruptions, others are evolving under evolutionary constraint. Constrained by what exactly? And how could this constraint be tested?