Making 3D Chess Feel Like Over-the-Board IRL

How do I make a 3D chessboard feel like playing over-the-board in real life? How should it balance between freedom of movement and game logic? Read my other blog post on the rendering for this chess game, which tries to make the imagery look as photo-realistic as possible.

Design Challenges

This online 3D chessboard is intended for beginning and intermediate chess players, and likely only Classic and perhaps Blitz games. The mechanics of moving pieces just doesn’t work well enough for the super-fast chess games, like Bullet, where pieces are moved in less than a second. I’m working on improving the game mechanics to try and make movement as smooth as possible to improve the feel and speed of playing in 3D. Below I talk about two new movement gestures that allow knights to jump over pieces and the new “capture & exchange” gesture that helps speed up play.

By far the most complicated logic in the app tries to allow the pieces freedom to move wherever they like, while maintaining the essential legal layout required if you are playing a game. Pieces are allowed to drift around their squares, you can knock them into each other and they fall over, and you can take captured pieces off the board. When the 3D positions and the logical game get out-of-sync, the app will automatically fix any issues, like removing captured pieces off the board, or finishing the rook side of a castle.

Oddly, the app code doesn’t actually know how to play chess. It has clients, one for Lichess.org and another for a local Stockfish webworker that know the rules of chess, maintain the legal state of the board, and validate and select moves for both players. The app asks the clients to make moves and listens for any illegal moves. The Stockfish client is especially smart, and can give us a list of the best moves, and their lines, so we can show hints and do analysis to support interesting visualizations.

Even though it doesn’t understand the full rules, the app does know how to detect a capture, including en-passant, castling, and how to promote pieces, because each of these special cases requires a “clean up” action, like removing the captured piece from the board, or moving the rook to finish castling. It also needs to be able to revert illegal moves, and a few other special actions, like tidying up all the pieces, in case some get knocked around.

Lichess Integration

Stockfish was not in the original app. The initial version required connecting with Lichess.org to play against their AI. That development choice shaped the design of the client interface, with the concepts of a set of active games, their UCI move histories, and the basic concept that the client is “in charge” of the game.

The design of the Lichess.org server API, especially the “Board” API, designed to allow you to connect real-world chess boards with Lichess guided my implementation, including the protocol used to communicate the new board positions.

When I added the Stockfish client, to allow people to play offline, using a local stockfish worker, rather than requiring online access, I just made it follow the existing interfaces designed by the Lichess.org server team.

Natural Piece Movement

Most chess games use a very computer-y interface, where you click on a piece to select it, then it highlights the squares for the legal moves which you can click on, and the piece moves. Often the piece just teleports instantly. Some 3D games use what I like to call “Wookie Chess”, where the moving pieces are animated and may even fight. This app avoids those mechanics and instead tries to feel as much like moving real pieces. You can move whatever pieces you want, whenever you want, and they try to stay where you leave them, rather than say, centering on their square automatically. If you're not careful you might knock a few pieces off their squares.

In order to balance between allowing you the freedom to move pieces wherever you want, and needing to keep the board positions in sync with the game logic, the app uses a concept of a “pending move” to keep track of secondary actions that need to happen to complete a move, like removing the captured piece from the board, or moving the rook after castling the king.

The idea behind the pending move is to give you time to move the captured piece or rook manually, rather than relying on the app to move it for you. The app waits as long as possible before forcing the required movement, until just before your next move, with the hope that you will manually remove the captured piece or move the castled rook.

If you sit down and watch a few videos of people playing chess, you will immediately notice a few things:

• Moves can be very expressive
• Beginners and advanced players move pieces differently
• Slides and jumps are the two basic types of moves
• Advanced players move and capture in a single motion

The initial version of the app allowed you to raise and slide the pieces one at a time. This resulted in lots of collisions, which is definitely part of the fun of this app. But when one user said “yeah, but I don’t knock over so many pieces when I play in real life”, I realized that the app was not mimicking real human motions, like jumping the knights. Recently I've added two new gestures to help make piece movement more natural: a single gesture capture maneuver that mimics what you would do on a real board with a single hand, and a smart jump used by a knight to clear over blocking pieces.

Single Gesture Capture Exchange

Capturing pieces requires two actions:
1. Dragging your piece onto the destination square
2. Removing the captured piece from the board

The initial version of the app required that you perform these two actions using two separate click-drag-release actions, one for your piece, and a second for the captured piece. If you failed to remove the captured piece, the app will remove it for you after the AI moves, just before your next move. You are free to do these two actions in either order. You can remove the captured piece before moving your piece:
https://youtu.be/jLNQWfxuoOc
Or, you can move it after, though this is pretty tricky since the pieces collide jostle around:
https://youtu.be/bSI8c44bfDU

The latest version v0.6.0, which you can play online adds a new “single action” capture gesture that tries to mimic how advanced players capture pieces. Now, when you drag a piece and collide with an opponent’s piece that you are about to capture, the app recognizes the situation, and drops the dragged piece and picks up the captured piece automatically, while you are still holding down the mouse button. Then you can continue with the gesture, dragging the captured piece off the board and releasing.

Now, with the new single-gesture capture-exchange, you can smoothly move your piece and then remove the captured piece from the board:
https://youtu.be/jb-M-GgfYLo

Jumping Knights

The previous version of the app does not have jumps. This was particularly problematic in the early phases of the game with knights that often have to jump over blocking pieces. Otherwise, they collide and send those pieces off their squares, and sometimes even off the board. There are some heuristics that disable collisions when you move pieces really fast, but you can’t rely on that when moving.

The new version 0.6.0 adds jumping for knights. If you click on the upper half of the knight, it detects which direction you are dragging, and uses its knowledge of potentially blocking pieces to raise it to avoid collisions, and otherwise allow it to slide through unobstructed space. This really cuts down on collisions, especially early in the game, when knights are blocked, keeping the board in good shape. However, since the piece moves upwards regardless of the movement of your mouse, there's a bit of a woozy disconnected feeling to the movement that I'm still tweaking.

https://youtu.be/vJbmEIjsu1w

Ideally, I’d love to find a natural movement UX that jumps when necessary, somehow without feeling a woozy varying distance between where you drag the mouse, and the relative position of the piece.

Next Steps

The next big feature to add to the game is multiplayer. Allowing two players to view and interact with the same chessboard at the same time - knocking pieces around together.

To establish a direct connection between two clients, we need to add a meeting room where the two clients can exchange contact information. I'm considering using the Lichess.org server to let people find each other to play games. That is, one player can be in the 3D app, while others may be on regular 2D boards in the Lichess apps. And, I may add a way to find players that are using the 3D board, though that might be tough since there probably aren't enough simultaneous 3D visitors to make that a good experience? We'll see.

Multiplayer requires adding a backend server, which is currently not required since everything runs locally in your browser. My plan is to implement communication using point-to-point WebRTC, with the initial connection and “meeting room” implemented as an ephemeral Cloud Run server, using session affinity.

Each client will run the full physics simulation locally, and the positions & rotations of all the pieces of one player are sent to the other and used to correct and synchronize the physics simulation. This allows smooth motion when running locally, insulated against network lag, and keeps both sides in sync with the same board. This is particularly easy when playing a game, since only a few pieces are moving at a time.

It will be fun to finally get this 3D chessboard working so I can play with friends.

---

Please join my 3D Chessboard Discord server to chat with me directly. Try out the 3D Chessboard online. It's free!