There are several rock cuts, so after a looking at a few options, I chose the ceiling tile method. The way that things work out, it was just as easy for me to use ceiling tile to build up the hills too. I used spackling compound (and wood filler in some locations) instead of plaster. It's a little more expensive, but when you consider the fact that this is a small layout, the difference isn't as considerable as on a large layout. Due to the size of the layout, and my method of doing things, I work in mostly small areas at one time. Being able to just open up a container for a little bit of compound every time I needed it was much much better than having to mix up plaster, and toss the unused portion. In fact, with all of the unused plaster that might have gone to waste, the spackling compound might not have been more expensive after all.
Working with ceiling tile is very dusty, so I decided to go ahead and do all of the ceiling tile at one time, so that I only have to clean up the mess one time. While it is very dusty, it is a fine dust that is very easily vacuumed up from practically every surface, unlike dust that is created when cutting up foam, or any "slop" from plaster or the like. So it's not the worst medium to work with. Just remember to wear a dust mask when you go to sanding it down.
To make the craggy rockface, I score the ceiling tile along where I want to cut it, place that line over the edge of a board or table, and "snap" it. There are many different articles and books where this is illustrated well, so I won't re-hash that here. For the sides of the hills, I simply stacked up the layers of ceiling tile like one would do with layers of foam. Then, I simply sanded the contours to my liking with 60 grit sandpaper on a sanding block. This took a little time and effort, but it came out well.