At my recent Oceana High School class reunion, seven of my classmates who had played football together lined up as they do each time we have a reunion—in poses that mimic the pictures of them in our high school yearbook. There are fewer of them now, and some of them are finding it harder to bend their knees these days. But each time we have a reunion, I enjoy seeing how they’ve changed and how they’ve stayed the same. This week they made travel plans to visit one of the coaches of their freshman team, who is in need of a heart transplant.

My classmates have big hearts, and they no longer fit the description of the cheer we chanted on the sidelines, “Ah, we’re mean! The Big Red Machine!”

I’ve thought of them often in the past two weeks. As a long time Washington fan, I thought of them as I watched my controversially named team lose its first game of the season. We were the Oceana Indians, a school that no longer exists, though each of our yearbooks has countless pictures of our mascot in full headdress on the sidelines. I’ve thought of them, too, as I’ve followed the controversy over whether NFL Commissioner Roger Goodell should resign in the wake of the Ray Rice controversy. I’ve heard some of my classmates’ stories of the indiscretions their coaches overlooked, stories—told with laughter—that would get such coaches reprimanded today.

My classmates are men of integrity. Though there may be such stories, I never heard them say that a coach overlooked something that would endanger them or the people around them. When they speak of their coaches, they speak fondly of men who led them to spectacular records but who also reminded them that sportsmanship and character were important long after their football careers ended. Football helped shape them into the men they’ve become.

Their school experience was distinctly different from my own. While my classmates learned to play football their freshman year, I immersed myself in a world of books because my father demanded that I get the education he didn’t have. He wanted a better life for me.

He was also an abusive father and husband.

I don’t know whether any of my classmates or my teachers noticed the welts I tried to cover when I came to school. But I do know that when my father beat us, our neighbors closed their doors and turned up the volume on their television sets. And the one time I confided in an adult that my father was violent when he drank, the adult—a member of the church I attended—told me that I should testify about my faith to my father and, if I were beaten, I should count it as a privilege to be beaten for the sake of the cross.

My classmates and I grew up in the same world, but the factors that shaped us were far more complex than their being on the football team and my growing up with domestic violence.

In all of the controversy surrounding Ray and Janay Rice this week, I am certain of only one thing: No man—or woman—becomes an abuser without a childhood that has somehow gone horribly wrong. Most abusers were either bullied or victimized as children.

That is the conversation that has been absent from the media storm about Rice. Both the abused and the abuser have suffered condemnation at the pens of journalists and pundits, most of whom have little understanding of the dynamics of abusive relationships. But I have not heard one of them ask what happened in Rice’s formative years to lead to that moment when he punched his wife in an elevator.

Even Leslie Morgan Steiner, a journalist who survived an abusive relationship to write a bestselling memoir about it—defended everyone involved in the Rice controversy, ending her op-ed in yesterday’s Washington Post with the assertion that “we should hold abusers—and no one else—responsible for the damage they inflict.”

I couldn’t disagree more. Were there adults in his life who turned a blind eye when someone bullied or beat him when he was a child? Did his high school and college coaches teach him that winning was more important than integrity, even if he had to brutalize his opponent? Has he abused other girlfriends before Janay? Did his handlers in the NFL turn a blind eye because he was making them so much money?

If we want to place blame, then every single person who suspected or bore witness to the acts that formed and cemented Ray Rice as an abuser bears some degree of culpability.

But more important than placing blame in this one high profile case, we should be asking what we, as a society, are going to do for the children who are highly likely to become the abusers of the future. And beyond holding Ray Rice accountable, what are we going to do to tend to his wounded psyche?

Our world is a violent place. And beating down the violent people who terrify us is not enough. For every abuser we lock up, there are dozens more in their formative years who will be the news of the future.

If the march from Al Queda to the more brutal ISIL should have taught us anything, it is that destroying perpetrators of violence will not ensure the safety of our women, our children, our world.

Destroying Ray Rice’s or Roger Goodell’s career may give us some momentary sense of justice. But what will we do for our children?