In a security control system, facial features are indispensable components to biometrics. An effective face recognition system is usually composed of facial features transformation, geometric analysis, and recursive training for deep learning networks in order to counteract the deceit from spurious targets. Nowadays design difficulty of face recognition arises from the dilemma of lower computing resource usage, power consumption, and delay time to install system ubiquitously in a building. In this paper, a chip-based face recognition is proposed by using semantic features to resolve aforementioned problem. The proposed system does not adopt tedious training processes but employs semantic features to achieve a miniature database and absolutely real-time processing speed. Our experimental results indicate that the proposed face recognition system can efficiently discriminate target from peoples faces only with economical resource usage on chip.