Experiments on the visual system of carnivorous mammals have revealed complex relationships between the geometry and statistical properties of the visual world, and the geometry and statistical properties of the primary visual cortex. This review surveys an extensive body of modelling work that shows how a relatively simple set of general-purpose neural mechanisms can account for a large fraction of this observed relationship. The models consist of networks of simple artificial neurons with initially unspecific connections that are modified by Hebbian learning and homeostatic plasticity. Given examples of internally generated or visually evoked neural activity, this generic starting point develops into a realistic match to observations from the primary visual cortex, without requiring any vision-specific circuitry or neural properties. We show that the resulting network reflects both the geometrical and statistical structure of the input, and develops under constraints provided by the geometrical structure of the cortical and subcortical regions in the model. Specifically, the model neurons develop adult-like receptive fields and topographic maps selective for all of the major local visual features, and realistic topographically organized lateral connectivity that leads to systematic surround modulation effects depending on the geometry of both the visual input and the cortical representations. Together these results suggest that sensory cortices self-organize to capture the statistical properties of their inputs, revealing the underlying geometry using relatively simple local rules that allow them to build useful representations of the external environment.