I don't know if it's worth it to drive from California unless you're stopping off in Moab on the way but we have some trails. I did a trail that I am told is called the Tank Trap out near Santa Fe this last Saturday. Socorro also has some good trails at Gordy's. If you coming by Las Cruces, there are lots of trails down there. At least that's what I'm told. I've seen pics and they have an off road event called the Chile Challenge down there every year. I think it's mostly hardcore stuff.
If you want to wheel real close to Albuquerque, there is the Rio Puerco. It has sand dunes and rock crawling for beginners to buggy stuff.
A couple of pics from the tank trap
I have few pics from Socorro but I was in the FJC so I won't bore you.