You can raise the prosperity of a town by making sure that its caravans reach their destinations, and by completing quests from its Guildmaster. Usually it is the case that the the player will not own a connected castle and village, unless they have large numbers of both villages and castles. The prosperity of a castle is affected by the prosperity of the village that is geographically attached to that castle, even if the village is not owned. The prosperity of each fief also affects the amount of taxes they produce. Towns earn the most base taxes, villages second, and castles the least. In Warband, you automatically receive taxes and do not have to visit to collect them.
Owning a fief allows you to collect taxes from the populace every week (taxes accumulate, so you don't have to visit every week). Once you have a fief, there are a few things to remember.