The identification of specific galaxy populations in large-scale spectroscopic surveys represents an essential yet challenging task, particularly for rare or anomalous galaxies that deviate from the typical galaxy distributions. Traditional methods based on template-fitting or predefining spectral features face challenges in addressing the complexity and scale of modern astronomical data sets. To overcome these limitations, we propose GalSpecEncoder-KB, a modular and flexible framework that combines deep learning with knowledge base retrieval to enable efficient and interpretable analysis of galaxy spectra. The framework integrates a Transformer-based feature encoder, GalSpecEncoder, pre-trained with masked-modeling strategy to capture semantically rich and context-aware spectral representations. By leveraging a Retrieval-Augmented Analysis approach, the knowledge base constructed from catalogs enables metadata retrieval and weighted voting for target galaxy identification. Using the Sloan Digital Sky Survey as a comprehensive case study, we demonstrate the capabilities of the framework for target galaxy search. Experimental results demonstrate the exceptional generalizability and adaptability across diverse galaxy search tasks, including identification of LINERs, Strong Gravitational Lenses, and detection of Outliers, while maintaining robust performance and interpretable spectral analysis capabilities.