Under the same spatial and temporal conditions, the quantum computing is superior to the traditional computing. Because a speaker's speech signal features the time-varying property and the randomness, its characteristic parameters also show high-dimensional characters and large changes in adjacent frames. This paper, based on the quantum information processing theory, takes a frame of the speech signal as a quantum state, and uses quantum logic gate circuits to construct the neural network according to the traditional neural network, and obtains an efficient clustering of the speaker's speech signal. A speaker recognition model is built and a method based on the quantum logic circuit neural network is proposed. This model has a large number of global attractors, and the method can use them to effectively reduce the complexity of the speech signal processing. Through simulations on a classical computer and a comparison with the BP neural network speaker recognition model, it is shown that this method not only can accelerate the convergence rate of the model but also has a better robustness with respect to the parameter changes. The system's recognition rate with the method proposed in this paper is 3.34% in average higher than that with the BP neural network method.