Energy retrofitting is paramount to reduce the use of energy in existing buildings, with benefits to the environment and people's economy. The increasing use of novel technologies and innovative methodologies, such as Terrestrial Laser Scanning (TLS) and Building Information Modelling (BIM), is contributing to optimise retrofit processes. In the context of energy efficiency retrofitting, complex semantic 3D BIM models are required that include specific information, such as second level space boundaries (2LSBs), material energy performance properties, and information of the Heating Ventilation and Air Conditioning (HVAC) system and their layout. All this information is necessary for energy analysis of the existing building and planning of effective retrofitting strategies. In this paper, we present an integrated (semi-)automated Scan-to-BIM approach to produce BIM models from point clouds and photographs of buildings by means of computer-vision and artificial intelligence techniques, as well as a Graphical User Interface (GUI) that enables the user to complete the models with information that cannot be retrieved by means of visual features. Information about the materials and their performance properties as well as the specification of the HVAC component is obtained from a database that integrates information from BAUBOOK, OKOBAUDAT and ASHRAE. The Scan-to-BIM tool introduced in this paper is evaluated with data from an inhabited two-storey building, delivering promising results in energy simulations.