Facetime (Prepare for some bad face puns)
Let’s face it, (rim-shot), often when doing rigging for mobile/tablet games (at least for now) it’s a practice at making things look as not-crappy as possible in an age where people are used to seeing some pretty high-end real-time graphics.
Whereas Mobile GPU tech has made leaps and bounds in the last few years, raising poly counts, high-res textures and normal maps, we’re often stuck with PS1-PS2 level joint counts. This usually means a fairly hard-limit of 45 skinned joints and 2 influences per vertex. After giving us just enough for a decent body with a few fingers in the hands melted together to consolidate, that gives us very little to work with for any kind of facial animation.
Back in the day when consoles and PCs were dealing with that sort of joint count, it didn’t quite matter that the face wasn’t moving. The characters all looked like their faces had gone through a belt sander because that’s all they could process across the board graphically.
But now because of aforementioned GPU advances, we have some high fidelity looking characters with dead faces. Not to mention modern consoles and PCs setting a benchmark. In Infinity Blade for example, among a few other things, faces have been smartly covered up to make a fantastic looking tablet game that doesn’t run at 2 frames per second.
However for Lexica, the game we were working on, we couldn’t necessarily do that, as our game was starring characters from classic literature. It might have come across as an odd choice to put a gas mask on Emma Woodhouse or a full-face helmet on Dr Jekyll. You can also combine that with the fact that the game and therefore a lot of the art was initially planned for more of a top-down and far away Diablo-esque camera angle, but the repercussions of that are a whole other ball of wax. So what’s a story driven tablet game to do? The decision was made to simply go with joints for blinking and rely mostly on body animation (and no, there were no polys for a separate lid from an eyeball, it was simply grabbing verts above the eye texture space and flitting them down fast, squashing the eye texture space). If I were there at the time (this call was made a long time before I joined the studio and project) I might have maybe fought for doing at least 1 joint for eyebrows, but hindsight’s 20-20. That’s especially so with remote disconnected hindsight. To the credit of our fantastic animator Kyle Kenworthy, it really did work pretty well in most cases. But there were a few characters modeled with a slight smile so as to seem pleasant that were then sometimes coming across as weird and/or disconnected from the dire situations unfolding around them, and a few really dramatic moments that just fell flat on their face (rim-shot) because of all our characters’ blank stares. There was also no fantastic voice acting to lean on, as all dialogue was text. It was a game about reading after all.
So in pre-production for our second season of the game, investigating animated faces a high priority. First off, we had to figure out how many joints we wanted to add to the rigs, and how much we could add to the rigs before our performance guy would beat us over the head with his chugging iPad.
So I took two characters that were at different ends of the stylization spectrum (Queequeg from Moby Dick, and Maid Marian from Robin Hood) for our humanoid characters and quickly created 4 levels of facial rig detail not including the blink joints: 10, 7, 5, and 3. There are some physical integrity issues that go on especially around the brows that kind of make me cringe a bit, but I tried to be a little more careful with that in actual implementation (more on that possibly in a later post), but sometimes the power’s just not there for subtlety. Mobile games, brah.
10 – left, right, and mid eyebrows; left and right outer cheeks; left and right sneer/levator labii; left right and mid lips
7 – left and right eyebrows; left and right outer cheeks; left, right, and mid lips
5 – left and right eyebrows; 1 joint for both outer cheeks; left and right lips
3 – 1 joint for both eyebrows, 1 joint for both cheeks, 1 joint for both outer sides of lips
NOTE – in order to be able to iterate faster, when consolidating joints downward, in lieu of re-skinning to combine sides, I just parented them under a group and moved them as one, in effect, making them move as one joint would. As such though, showing the joints wouldn’t be accurate to what’s going on, just the controllers which should just be seen as visualizations of the joints, so it might be slightly hard to see what’s going on visually
So 10 obviously gave the most power and control mixed with some nice subtlety especially around the nasal labial folds.
7 saw only a little bit of drop considering it was a 30% decrease, definitely phasing out the need for a mid eyebrow joint and turning nasal movement in to more of a luxury item than a necessity.
5 still looked pretty good, but there was a noticeable lack of control over the middle of the lips because blending in to 50% weight split between the left and the right meant any rotation from the sides would make the middle fly off (that 2 influence limit rearing its ugly head.)
3 actually kind of surprised me. The difference between the 10 joint rig and the 3 joint rig really wasn’t anywhere near as bad as I was secretly thinking it might be. The pitfalls were there though. Though I was pretty proud of my idea to switch the pivot from one side to the other to fake isolated side movement, the lack of ability to ever counter rotate sides started to hurt, and there was no control over the middle of the mouth. Also, the big problem is having things go away from each other as they would in a smile for instance (both the sides of the lips and the cheeks). The best solution is scaling the joints, but a lot of engines don’t support that. Unreal supports uniform scale (at least last time I used it, that was the case) and Unity supports non-uniform scale in joints. We’re using unity, so there’s that, but for the purposes of the test, I wanted to use only rotate and translate, so I faked it as much as I could by moving up and forward or backward.
Spurred on by the success of the 3 joint rig, I decided to do another quick test on Marian, seeing how much I could pull off using lots of middle joints that controlled both sides, but in a higher joint-count situation. It’s a bit of an eyeful since it’s all crammed in the middle. I actually moved some the controllers out, so I could tell what was going on while animating it.
It’s 7 joints – 1 for both brows, 1 for the middle-brow area, 1 for both outer cheeks, 1 for both sneer areas, 1 for outer lips, and one for the middle of the mouth.
It’s a little unorthodox, but I think it was pretty successful. I was especially a fan of the cheeks and sneer ones as I felt they brought of cheap physical grounding in an area that doesn’t need as much fine control as the eyebrows and lips.
After sending these around to the team, the 7 joint count was set. I was a big proponent of doing basically the first 7 joint setup, but replacing the 2 cheek joints with the cheek and sneer setup from the last maid Marian setup as I thought it would make the best look. I gave the animator both 7 joint setups to play with but the he worried about the lack of easy isolated control on the sides, so since he was going to kind of be the one who would ultimately have to live more with the decision, I quickly deferred to him on that one.
…Which kind of reminds me of a little philosophy thing if I might digress from my already very long post (my bad), I was reading fantastic article in a monthly email by Animation Mentor co-founder Shawn Kelly a few years back that always comes to mind in these situations. The post was simply titled: “You Are A Tool.” Long story Short (I think we’re well beyond that with this post), he talks about how in the end, animators are all tools of the director/project. While it’s worth it to let your opinion be known, your opinion can and probably will be superseded many times during your career. You’ll work with people you disagree with, but thing is, it’s bigger than you and ego. Fact is, while you’re getting paid for your vision, in the end, it’s to push some one else’s vision; if it’s not their vision, it’s wrong. That’s kind of hard truth of any business, which in the end, is what this all is. We do it anyways, because it’s still fun as hell. I’m kind of doing the article no justice, but while it was aimed towards animators, it obviously applies to just about any job ever. And if the artist is the servant of the director/project and the tech artist the servant of the artist, we’re the servants of the servants. In the end, our job is to equip the artists do their jobs as well as they can. We can let our opinions be known, but ego has no place when a call is made. We’re tools who create tools for tools. The tools we create for those tools give them the tools to be better tools. Sorry, couldn’t help myself.
Anyways, I hope you survived through the post, and that it was informative and helpful. Luckily for us, we had it in our hardware budget to improve by more than 3 joints, but for some projects, maybe 3 will be all that’s actually available. Next post on this subject I’ll talk about how some of the facial stuff was actually implemented based on all this experimentation.