It is not uncommon in modern liturgical celebrations at your typical American Novus Ordo parish for a vigorous round of applause to be granted for a homily or, more common, the musical performance of the "praise band" or other musical "performers." Is this an acceptable practice?
It must be admitted that in history this did occasionally happen. I remember reading a book of sermons by St. Augustine in which it was actually recorded in the sermon (apparently transcribed by an acolyte) that the great Bishop of Hippo was sometimes interrupted several times during his sermons by applause and accolades.
That being said, today is not the ancient Church, and centuries of Christian tradition have generally frowned and that sort of spontaneous applause at Mass. For one thing, it is not even spontaneous anymore; parishes that clap typically do so in an almost ceremonial manner, where it always follows the closing song regardless of what it is or how skilled the musicians are. So it is really laughable to speak of a "spontaneous" applause.
Is this practice outlawed? It is not outlawed explicitly in any rubrics or codes, which is why I answer that it is not definitively authoritative. Yet even if not specifically proscribed, such acts go against the spirit of sobriety which characterizes the Roman liturgy. This is especially so in that it makes the liturgical celebrants and participants into entertainers who merit applause because they demonstrate physical skill (witty sayings in a homily, talented musical performance) rather than because they are worshipers offering something to God. A liturgy in which applause breaks out must of necessarily be a profoundly man-centered affair, something where we are celebrating ourselves, not God.
No one less than our current pope, Benedict XVI, makes the same statement when he commented on the practice of applause at Mass in his 2000 book Spirit of the Liturgy:
"Whenever applause breaks out in the liturgy because of some human achievement, it is a sure sign that the essence of liturgy has totally disappeared and been replaced by a kind of religious entertainment. Such attraction fades quickly - it cannot compete in the market of leisure pursuits, incorporating as it increasingly does various forms of religious titillation." 
Applause at Mass signifies that the truth about what the Mass is has been lost. It is contrary to Christian Tradition, to the theology of the liturgy, to the wishes of the pope, and consequently ought not to be endorsed or encouraged.
 Pope Benedict XVI, Spirit of the Liturgy (Ignatius Press: San Francisco, 2000), 198