Under the assumption that security price follows the Geometric Brownian Motion, this paper studies the investment problem with transaction costs in a n security financial market, by using the theory of stochastic optimal control, and an optimal control model is established for the problem of security investment with transaction costs. First, this paper states related theory on stochastic optimal control, and provides the definitions of value function, utility function and viscosity solution.solution．Secondly，in terms of viseosity solution，a PartiaI INferential Equation(PED) is derived for the value function in two cases!bounded trading rates and unbounded trading rates．The obtained PDE is a free—boundary problem governed by avariation inequality．Finally，the optimai investment strategy is provided based on the value function of the stochastic optimal contro1．The result of this paper is userul in such practice as fund management，financial risk management．It is hope d that the resuhs can improve the scientific ievel of decision making．