Sometimes there is nothing else you can do... my Client wanted to replace a failed shrub by their front gate, which is at the top of a steep flight of steps down to the house.
So I scrabbled around in the undergrowth, and found quite a pile of old bricks and half-bricks: I suspect that someone had already had this idea, but so long ago that the whole pile had disintegrated.
I dug into the slope about a foot and a half lower than where I planned to plant, levelled off a curved edge, and built up a simple wall of the old bricks. There was no mortar, obviously, they were dry-laid, which naturally left plenty of gaps for the water to dribble gently down and out, without building up behind the wall.
You might have noticed that "proper" brick or stone retaining walls always have a small pipe coming out somewhere near the bottom, for this exact reason.
Having built my little wall, I packed the soil in behind it, and filled in with compost, which created a nice flat platform for the new shrub. It doesn't look particularly pretty right now, but I added a couple of tumbling annuals - alyssum, lobelia - at the edge, and they will soon cover it up.
I also made a point of arranging the soil to slope the "wrong" way, so that when the Client watered the shrub, the water would be caught by the soil, and would not simply run off the edge of the newly created platform. That, along with strict instructions to water slowly, should do the trick.
In time, the stones will disappear under the new growth.
In fact, this particular one had to be re-done: you might notice that the water has escaped through the gap, so I had to add another stone to hold it back.
In all these cases, I must repeat that water SLOWLY is the key factor in getting plants to establish on a bank.
If you have the option, running some perforated seep hose along the area is a quick way to do it, but there is not always a tap close enough. In this garden, all the water is at the bottom, and all the planting is way, way uphill!
But I like a challenge.