Check out this nifty little touch-screen-compatible, WebGL-powered application.
To test out the latest AI, I added GitHub Copilot to VSCode and asked it to build a simple web application that lets the user move three radiant lights (red, green, and blue) around a screen to see how adding colors works. (For example, if the three colors are right on top of each other it looks like a single white light.) Here’s a screenshot of that first app:
By default Copilot uses GPT-4o, but on a few examples I have found that Claude 3.7 Sonnet (another Copilot option) is capable of more sophisticated computer engineering, so with that selected as my Copilot “Agent” I began enhancing this app. The most significant change – and something I’ve wanted to try for a while – was to use WebGL to take advantage of the graphics processing features built into most modern electronics. Thanks to that hardware acceleration, this enhanced app supports lots of light sources, dithering to avoid color banding, and real-time dragging lights around the screen without noticeable lag. Then I added touch-screen support so that the app can be used on mobile devices.
It took some coaching from me to get this working: At several points I observed bugs and Copilot would essentially get stuck in a loop saying, “Oh, I see what’s wrong; this should fix it,” without successfully fixing it. I had to guide the Agent through more intentional debugging methods to resolve several confusing problems. But by the end I hadn’t written or even touched much of the code. I was the designer and tester, and Copilot saved me the trouble of:
- Scouring API documentation and sites like StackOverflow for code samples needed to make it work.
- Learning or remembering the exact syntax of the languages involved (WebGL, JavaScript, CSS, HTML).
- Recreating common GUI tricks, like adding code to make sure that everything is visible on a screen regardless of its size or orientation.
- Finding and fixing minor bugs.
- Writing debug code to understand and resolve major problems.
Here’s a screenshot from the final app (shown here with all light inverted – one of the fun features accessible by right-clicking/long-tapping):
