Fractional order controllers have a growing popularity in last years and they give more flexibility to researchers for designing a controller. In this paper, a fractional order PID controller is designed for the position control of a two wheeled inverted pendulum. The pendulum is modeled with DC electrical motors to obtain a more realistic model. Integer order PID controller is also designed to make a comparison with fractional order controller and all controllers are optimized by swarm algorithms to be sure obtained the best performance for each controller. A fractional order PID controller has two extra parameters (lambda and mu) and totally five parameters to be optimized. Optimization algorithms are powerful tools for designing a controller, and guarantee finding an optimum result. However, each optimization algorithm has a different performance not only because of the structure of the algorithm but also depending on the optimization problem. Because of this, four popular optimization algorithms (artificial bee colony, particle swarm optimization, grey wolf optimizer, and cuckoo search algorithm) are used to tune controller parameters, and compared regard with the optimized system performance. The results show that the best performance is obtained by the fractional order PID controller, which optimized by artificial bee colony algorithm. The fractional order PID controllers have also better performance than integer order PIDs when used the same optimization algorithm for tuning.