Recent concerns about potential biases against novelty in science and a perceived decline in creativity have raised interest in quantitatively measuring novelty in research outputs such as scientific articles and patents. However, defining and measuring novelty is a complex task, as it involves capturing the elusive spark of creativity, which is inherently unobservable. This paper proposes a systematic approach to tackle these challenges. First, we propose a theoretical framework to conceptualize the novelty, thereby offering clarity in its measurement. Subsequently, we review existing indicators, categorizing them into families according to our framework and their fundamental principles. Additionally, leveraging data sourced from OpenAlex and PATSTAT, we systematically evaluate the performance of these novelty indicators using a dataset comprising scientific articles and patents filed at the United States Patent and Trademark Office between 1980 and 2017.We conduct validity tests to assess the ability of indicators to capture non-random signals and test convergence validity within and between indicators’ families. We also evaluate the capacity of novelty indicators to capture strong newness signals, such as Nobel Prize discoveries. The preliminary findings highlight the complementarity rather than convergence of different indicator families, emphasizing their distinct contributions in capturing various facets of novelty. Our contribution includes a navigational framework for scholars and insights into robust novelty measures crucial for informed policy evaluations and funding decisions in science and technology.