The evaluation of indoor risks is a paramount issue in building design and construction. Conventional methods that rely on handcrafted rules or drills are insufficient for this task as they either fail to accurately depict the sophisticated spatial attributes and people's cognition abilities or are not suitable during the design phase of the building. This paper puts forward a novel computational framework with a reinforcement learning-based paradigm to automatically assess the evacuation risk posed by the indoor space through intelligent agents. Our model focuses on the agent's exploration behaviours as it gains knowledge to locate the optimal path from an initial location to an exit. The cost of this knowledge acquisition process is then used to capture the risk posed by the initial location of the building. The work aims to shed new light on utilizing agent-based techniques in evaluating building safety.