You can setup handbrake in a jail, share your storage containing your DVD images with the jail, and script the transcoding with handbrake. If I recall correctly, and if the state of affairs hasn't changed over the last few years, you have to install handbrake from ports for it to work correctly.