I argue that Husserl's notion of horizon and Searle's notion of background offer a contextual model of perception that significantly reformulates the debate about the conceptual vs. nonconceptual content of perception. I illustrate the model by using a test case: the perception of an ancient Roman milestone—an example given by Husserl—which both Husserl and Searle consider to be a direct and immediate perception without inferences involved. I further differentiate Husserl's and Searle's views, arguing that Husserl's model has the advantage of accounting for the diachronic aspect of perception.