This article raises the possibility of speaking of a lithic archive among the indigenous nations of the Andean region. This concept refers to a form of memory configuration that interweaves ancestral narratives, stones with singular characteristics, and territory. A hermeneutic reading will be made of two of the fundamental texts of seventeenth-century Peruvian viceregal literature: Comentarios reales by the Inca Garcilaso de la Vega and the Manuscrito de Huarochirí, by an anonymous author. Both documents provide several examples that suggest the importance of certain stones in the transmission of knowledge in pre-Columbian times. The lithic archive must be understood within a larger ritual framework that highlights the importance of territory for indigenous memory.