GPT-3/Codex: An AI that can write Minetest mods... kinda

Post Reply
User avatar
rubenwardy
Moderator
Posts: 6660
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

GPT-3/Codex: An AI that can write Minetest mods... kinda

by rubenwardy » Post

Image

An exploration of GPT-3's knowledge of Minetest modding, with node registration, formspec manipulation, mod creation, and Minecraft mod to Minetest conversion.

Read it on blog.rubenwardy.com
Renewed Tab (my browser add-on) | Donate | My Twitter | Mods | Minetest Modding Book

Hello profile reader

User avatar
ronoaldo
Member
Posts: 153
Joined: Mon Dec 07, 2020 01:04
GitHub: ronoaldo
IRC: ronoaldo
In-game: RonoaldoKakashi
Location: São Paulo, Brasil
Contact:

Re: GPT-3: An AI that can write Minetest mods... kinda

by ronoaldo » Post

This is was an amazing reading. Thanks for sharing and doing that experiments!
Servers: Mercurio | Tools: ContentDB CLI | Mods: minenews

User avatar
Blockhead
Member
Posts: 444
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
In-game: Blockhead
Location: Land Down Under

Re: GPT-3: An AI that can write Minetest mods... kinda

by Blockhead » Post

We live in an era where we may soon be having people with no Lua knowledge who asked GPT-3 to write a Minetest mod for them, and then coming to the forums to ask us why their code doesn't work. Astounding :)
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My mod: Foodblocks

User avatar
sorcerykid
Member
Posts: 1699
Joined: Fri Aug 26, 2016 15:36
GitHub: sorcerykid
In-game: Nemo
Location: Illinois, USA

Re: GPT-3: An AI that can write Minetest mods... kinda

by sorcerykid » Post

I actually found this to be both intriguing and amusing, simultaneously. Very insightful!

User avatar
v-rob
Developer
Posts: 934
Joined: Thu Mar 24, 2016 03:19
GitHub: v-rob
IRC: v-rob
Location: Right behind you.

Re: GPT-3: An AI that can write Minetest mods... kinda

by v-rob » Post

Oh no, AIs making brittle formspec code for Minetest. We're doomed! Please, nobody teach them how to make forum accounts... But actually really interesting.

Gosh, though, IDEs are getting more gross all the time. Now they're flat out suggesting code? I get that boilerplate code is annoying and prone to mistakes and all, but it's worse to have barely competent programmers just using brittle and possibly wrong code written by an AI.
Core Developer | My Best Mods: Bridger - Slats - Stained Glass

User avatar
rubenwardy
Moderator
Posts: 6660
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

Re: GPT-3: An AI that can write Minetest mods... kinda

by rubenwardy » Post

Thanks for the comments

Just an update - in the article, I mention that the fruit list formspec it makes has an error due to a missing variable storing the list.

I passed GPT-3/Codex the error message and it fixed it. This system could be very powerful if combined into some sort of feedback loop, where it can receive error messages / run tests

Image
Renewed Tab (my browser add-on) | Donate | My Twitter | Mods | Minetest Modding Book

Hello profile reader

User avatar
rubenwardy
Moderator
Posts: 6660
Joined: Tue Jun 12, 2012 18:11
GitHub: rubenwardy
IRC: rubenwardy
In-game: rubenwardy
Location: United Kingdom
Contact:

Re: GPT-3: An AI that can write Minetest mods... kinda

by rubenwardy » Post

v-rob wrote:
Thu Jun 23, 2022 23:19
...
I'll repost what I replied to someone else:
It's about as good as a beginner modder.

If you look in the formspec section, it generates complete code for a problem that almost works - just has one missing variable. If you feed it the error message, then it is able to fix the issue - just like a real programmer does

So if you could combine this tool with a way of testing its results then it will be very capable

Remember that this is an early generation of the tool as well. I fully expect subsequent tools to be much more accurate.

Also, this tool was never trained especially for lua and Minetest. Codex only really officially supports python, js, and a few other languages. So having it trained especially will make the results better
Renewed Tab (my browser add-on) | Donate | My Twitter | Mods | Minetest Modding Book

Hello profile reader

User avatar
DS-minetest
Member
Posts: 1214
Joined: Thu Jun 19, 2014 19:49
GitHub: Desour
IRC: Desour
In-game: DS
Location: I'm scared that if this is too exact, I will be unable to use my keyboard.

Re: GPT-3: An AI that can write Minetest mods... kinda

by DS-minetest » Post

Interesting!
Can we throw this on feature requests?
ruben's atricle wrote: GPT-3 (and Codex) learned how to write Minetest code by reading code on the Internet. This code may or may not be open source, and may or may not be permissively licensed. These models tend to regurgitate code, which leads to license laundering - open source code being turned into proprietary code, without credit. These products benefit from the unpaid labour of the open-source community. So whilst this technology is interesting, I’m not sure how much I agree with it ethically.
This sounds quite problematic.
If the AI outputs non-trivial material (let it be code, text, or any other media), you can't use it (especially not without giving credit). It's like you'd have found the material somewhere in the web. But you can also barely find out where the material even came from, and who to give credit.
Note that I've recently renamed myself on github and co. to "Desour". (I'm bad at naming things.)
Feel free to call me DS.

User avatar
Blockhead
Member
Posts: 444
Joined: Wed Jul 17, 2019 10:14
GitHub: Montandalar
In-game: Blockhead
Location: Land Down Under

Re: GPT-3: An AI that can write Minetest mods... kinda

by Blockhead » Post

I was thinking about the licence laundering issue. While we can require anybody to do this, perhaps we could expect that people who write code with GPT-3/codex and want to be ethical about it to also ask it "Is the above code copyrighted?". Sadly nothing can likely be done about people who don't want to be ethical. Also this is more of an issue for copyleft than permissive licences.

I think/hope that in the future, the fact that only source-available/open source code will be usable in improving AI models, coupled with increased use of AI programming techniques, may mean more companies will all agree to share their source in order to improve the overall models for everyone. At least that's something to be hopeful about, if not really guaranteed.
/˳˳_˳˳]_[˳˳_˳˳]_[˳˳_˳˳\ Advtrains enthusiast | My map: Noah's Railyard | My mod: Foodblocks

User avatar
Mantar
Member
Posts: 359
Joined: Thu Oct 05, 2017 18:46
GitHub: jeremyshannon

Re: GPT-3: An AI that can write Minetest mods... kinda

by Mantar » Post

The laundering problem is best solved by training an AI solely on public domain/permissive licensed material. If you include GPL code, then any code it generates should be treated as GPLed, which would not be a problem for us here.
Attribution licenses would have to be excluded, there's no way to figure out what the AI borrowed from where and include the necessary copyright notices.

Post Reply

Who is online

Users browsing this forum: rubenwardy and 3 guests