I'm going to approach this from an idealistic standpoint. That is, let's imagine that the technology to perform such an operation were perfected, and the healthy areas of a person's personality, including everything that enables them to function everyday, were retained, while the negative sections were excised. So, they would be the same person minus everything that was bad about them. No doubt, this would still be an alteration to that person's personality, so one could still compare this method to the death penalty even in the ideal scenario.
We have to assume that everybody knows about this person's crime, and that they would eventually find out about it, as well. Having no memory of committing the crime, the person who used to be a criminal can be said to be 100% innocent, because it was the other, malevolent personality who committed the crime. They would walk around believing that other person they used to be was the guilty one, and they are innocent, while everyone who sees them would give them a wide berth when they see them walking down the street. They would therefore be a pariah, and suffer punishment for a crime they believe someone else committed, someone who was erased from existence.
There's also the matter of whether this person retains any meaningful lesson after having their memory erased. When I think about stupid things I've done in the past, and consider whether I would prefer if they were erased, I decide that I would not, because without those memories and the lessons they imparted to me, I might make those same stupid mistakes again. If a person knows intellectually that a crime is bad, but has no personal, negative emotion to support that idea, it doesn't quite have the same restraining effect which would prevent them from committing the crime. For example, when I was little, I stole a Kit-Kat bar from a store when my mom wouldn't buy it for me. Then I faced the consequences of that action, and never thought about stealing another candy bar ever again. What if my memory of the crime were erased? I think I'd have another candy bar.
This also brings up an interesting idea related to religion. For those who believe that a person is judged when they die and are either rewarded or punished for their deeds, how would such a person be judged? It's not the same person who committed the crime, but not in the sense that they atoned for their crime themselves, since they have no memory of it, and would not be able to genuinely make atonement. Maybe the soul would be split, and one part would be judged separate from the other?
I don't think this would be a step in the right direction. People undergo personal growth by learning from their mistakes, which is impossible if they can't remember what those mistakes were. Plus, this isn't a form of justice if the person who suffers from alienation from society isn't the same person who committed the crime.