To support dramatically increased traffic loads, communication networks become ultra-dense.Traditional cell association(CA) schemes are timeconsuming, forcing researchers to seek fast schemes.This paper proposes a deep Q-learning based scheme,whose main idea is to train a deep neural network(DNN) to calculate the Q values of all the state-action pairs and the cell holding the maximum Q value is associated. In the training stage, the intelligent agent continuously generates samples through the trial-anderror method to train the DNN until convergence. In the application stage, state vectors of all the users are inputted to the trained DNN to quickly obtain a satisfied CA result of a scenario with the same BS locations and user distribution. Simulations demonstrate that the proposed scheme provides satisfied CA results in a computational time several orders of magnitudes shorter than traditional schemes. Meanwhile, performance metrics, such as capacity and fairness, can be guaranteed.