Check out ncmpcpp.
ncmpcpp
Mopidy is specifically designed for that, with a bunch of plugins api etc.