A lot depends on the amount of timber you have available, and whether you are in it for the long haul, or just want to make some money off a limited amount of timber. The more you can process the wood, the more money it will bring you. If you can supply kiln dry lumber, you can charge about double the price of lumber straight off the mill. Even air drying will help increase the value.
It depends somewhat, too, on the size of the trees and the support equipment you have. For example, you can do pretty well with ties IF you have the right equipment, including a hydraulic mill and a loader that can pick up bundles of 10 (20 is much better). Check with Koppers for specs. They buy tie lumber for their treating plant. Also check for any flooring mills and cabinet shops in your area.
If there isn't too much competition in your area, trailer decking, barn siding, and fencing can be a pretty good deal, especially for starting out. Set the nice stuff (like quartersawn oak) aside for the woodworkers who will pay a premium for it. I've never worked with a broker, but assume they like to work with larger quantities of wood than I produce.
There is no sure-fire way to make money with a mill, but it can be done. The small mills that do turn a profit do so by hard work, good customer service, and careful money management. A manual sawmill can be good to start out part-time, and a lot of people run them full time, but hydraulics more then pay for themselves in production operations.