Photonic quantum computing is a leading approach toward universal quantum compu-tation. Here, we propose a realistic model for the implementation of neural networkson photonic quantum computers. Specially, we design a quantum circuit built inthe continuous-variable (CV) architecture that encodes information in the spectralamplitude functions of single-photons. This circuit consisting of some electro-opticalmodulators and XOR boxes that, respectively, adjust and combine their entries to pro-vide a weighted sum of input signals. We show that our model can reproduce classicalneural network models while maintaining some quantum phenomena such as superpo-sition and entanglement. In particular, we utilize the circuit as a quantum classifier andvalidate the CV quantum neural network architecture through doing some machinelearning modeling experiments. Such a quantum circuit can be implemented on theCV photonic quantum computers that promise exponential speed-up over the classicalcomputers for specific tasks.