Inspired by a certain reading of Wittgenstein , especially his notion of rule-following , this paper sketches an outline of an idea of elementary mathematics (arithmetic and basic geometry) as an intrinsic part of intellectual formation, or Bildung . In particular, I argue that we need to distinguish between mathematics as an activity and mathematics as a body of knowledge, where it is the first mentioned that is primary for the character of thinking. My claim is threefold: (i) The development of the capacity to think mathematically, together with the ability to read, write and speak one's native tongue with clarity and precision, ought to be the primary aims of primary and secondary schooling; (ii) At this basic level, the capacity to think mathematically is inseparable from the capacity to reason in general and should be seen as an essential part of the latter; (iii) These two claims, if correct, have profound consequences for how we ought to think about the form and content of teaching .