Lovehurts, I am a U.S. attorney.

"Adultery" is actually an old criminal cause of action that the state might file against a philandering spouse; it is not in most jurisdictions a civil action you can "file" yourself against a spouse.

In some states, there is cause of action in civil court that your can file sometimes called "loss of consortium."  This is a claim that as a result of someone's conduct there was injury to your marital relationship and you demand legal damages from the "other woman."  It is pretty rare to prosecute these kind of cases, however, especially between consenting adults in the classic affair situation.  In fact, many states no longer even recognition this kind of lawsuit anymore.

Of course, you can file for divorce against your spouse, using adultery as legal grounds for getting it.  And depending on the laws of your state, the fact of the adultery may have some influence in terms of the disposition of property and/or child custody.  However, most states nowadays are considered "no fault" divorce states in which the conduct of either parties is not really considered relevant for most issues of the divorce.  It is only a small factor in practical terms among many others.

For what it's worth...