AI in Game Development: Gamedev Tips, Tools, Techniques, and GPT / LLM Agent Integration

I am currently heading towards a video call avatar and I thought some of the stuff from that research might fit well for gaming as well. I mean most of it was made for gaming anyways…

Here is a fresh one:

Apple Machine Learning Research

HeadsUp: Large-Scale High-Quality 3D Gaussian Head Reconstruction from...

A scalable feed-forward method for reconstructing high-quality 3D Gaussian heads from large-scale multi-camera setups. Trained on 10,000+ subjects with state-of-the-art reconstruction quality.

And you should also watch out for the work of Olga Sorkine-Hornung and her work on:

Laplacian Mesh Editing

igl.ethz.ch

igl | Interactive Geometry Lab | ETH Zurich | Laplacian Surface Editing

Abstract, paper, video, conference presentation and MATLAB demo.

and especially

ARAP: As-Rigid-As-Possible Surface Modeling

igl.ethz.ch

igl | Interactive Geometry Lab | ETH Zurich | As-Rigid-As-Possible (ARAP)...

Abstract, paper, video and other publication materials.

That stuff is patented by Disney afaik… so you may be careful how you use it - but on the other hand it is good to know none the less.

You can connect your codex to blender (3D modelling tool) and then work on a so called mesh*

Avataaaaaar

Here are a few more words to feed chatgpt with:

landmark extraction, pose control, mimiks, phenoms and shape keys.

In that context adding that little background information about those techniques make alot of sense.

Have fun