Python as a modding language
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
Oh, and the idea that Python constrains your choice of object-orientation is also not true. Python's metaclasses and custom attribute access, for example, make prototyping pretty damned easy if that's what you want to do. At least as easy as hacking other forms of object-orientation into Lua.
- v-rob
- Developer
- Posts: 971
- Joined: Thu Mar 24, 2016 03:19
- GitHub: v-rob
- IRC: v-rob
- Location: Right behind you.
Re: Python as a modding language
But Python and Lua together? That's a big problem, and it isn't worth the trouble and confusion. Plus, the devs aren't going to do it because Python is unnecessary because Lua works just fine, all mods are made in Lua, and they already have enough to do.
Re: Python as a modding language
Can we change the poll option "yes" to "yes but I understand it will never happen" ?
Re: Python as a modding language
+1!sofar wrote:Can we change the poll option "yes" to "yes but I understand it will never happen" ?
And add an Ada option, just for kicks.
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
Eh. I certainly don't expect it'll happen in the actual game, given the history and attitude of the development team. But someone might be interested in doing it in a fork or in a similar project, so it's still pretty useful info.
Re: Python as a modding language
Did you run the python code on multiple cores? If yes, then YOU don't make an apples-to-apples comparison, because you removed the optimization on one language, but leave it to the other, and this makes the performance of LuaJIT even more impressive. An unoptimized Lua code running on a single core is still faster than an optimized python code running on multiple cores.prestidigitator wrote:So in an apples-to-apples comparison Python is about 18% slower (66 seconds vs. 56 seconds) for this particular benchmark.
This "benchmark game" tries to compare languages in a certain way. Everyone can try to write optimized code for each language and compare with each other. In this case, Lua tries to optimize with caching, python tries to optimize with multicores.
You are right, but I don't think that there will be much difference.prestidigitator wrote:In addition, none of these benchmarks compare various methods of transitioning between C and the particular scripting language in question, and that is likely to be far more interesting in the Minetest case than shuffling some array elements around like in the benchmark you chose. Comparing similar tasks involving bidirectional calls and data movement between the Lua API and, for example, CFFI for Python would be an interesting exercise. Maybe I'll play with that at some point.
There is a reason why Python is not used in most games. You can't use the benefit of Python. You need performance. And it must be safe (no/restricted OS access)
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
I literally converted the Lua code to Python, line-by-line. It ran on a single core. The code is right there.burli wrote:Did you run the python code on multiple cores? If yes, then YOU don't make an apples-to-apples comparison, because you removed the optimization on one language, but leave it to the other, and this makes the performance of LuaJIT even more impressive. An unoptimized Lua code running on a single core is still faster than an optimized python code running on multiple cores.prestidigitator wrote:So in an apples-to-apples comparison Python is about 18% slower (66 seconds vs. 56 seconds) for this particular benchmark.
Depends on how it is being used. And on how it is sandboxed. The same thing applies to Lua. Just a little while ago, you could destroy anything in the filesystem which the user running the game had write access to, not to mention pulling all kinds of confidential info from files and sending it over the network, all from a Minetest mod.burli wrote:There is a reason why Python is not used in most games. You can't use the benefit of Python. You need performance. And it must be safe (no/restricted OS access)
I mean, to be fair Lua isn't used in most games either. I'm not going to argue against the fact that Lua is very small and simple, and relatively easy to sandbox. However, the more of the application or game you implement in your scripting language, the more useful it is to make it a robust, mature, featureful language.
As for the performance argument, it's outdated. If you need highly optimized performance in Lua, you use LuaJIT instead of the default interpreter. If you need highly optimized performance in Python, you use PyPy instead of the default interpreter. As the benchmarks you pointed to show, default Python is sometimes faster than default Lua (yes, if you take into account the number of cores each test uses for each implementation). Similarly, it's likely that optimized Python is sometimes faster than optimized Lua. Interpreters and (JIT) compilers mature, and performance improves. Your stereotypes should as well.
- Linuxdirk
- Member
- Posts: 3219
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: Python as a modding language
And even if you proved by yourself that even under optimal circumstances the same stuff is slower on Python you still fight for Pythoin to be implemented (what NEVER would happen, and you know that)?prestidigitator wrote:I literally converted the Lua code to Python, line-by-line. It ran on a single core. The code is right there.
Except most scriptable/moddable games use it.prestidigitator wrote:Lua isn't used in most games either
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
Um, no?Linuxdirk wrote:And even if you proved by yourself that even under optimal circumstances the same stuff is slower on Python you still fight for Pythoin to be implemented (what NEVER would happen, and you know that)?
- Under optimal circumstances, ONE benchmark was about 20% slower in Python, when someone was trying to make it out to be orders of magnitude slower.
- Not fighting for it to be implemented, actually. Pointing out flaws in reasoning used to argue that it should never be implemented. There's a difference. As I said: prestidigitator wrote:Eh. I certainly don't expect it'll happen in the actual game, given the history and attitude of the development team. But someone might be interested in doing it in a fork or in a similar project, so it's still pretty useful info.
Sure. Python hasn't historically been a very fast language, and games typically have a lot of history to contend with, in libraries, their own code base, developer misconceptions, etc. This is like the argument that most games are written in C, and therefore no game could ever be written in anything else because no other language is good/fast enough. It's a historical argument made by people whose conception of performance is stuck in the past. New languages are being developed. Older languages are maturing and being made faster. Technology moves on, and progress is made. That's a good thing.Linuxdirk wrote:Except most scriptable/moddable games use it.
Does that mean that you must use Java or Python or whatever for your game? Of course not. But you're conflating choice with necessity, and that's counterproductive in the long run. You folks honestly don't have to make these (false!) arguments and get defensive about them being challenged in order to ENSURE Minetest will never, ever, ever use Python. What exactly are you so worried about?
- ThomasMonroe
- Member
- Posts: 286
- Joined: Tue Apr 04, 2017 16:21
- GitHub: ThomasMonroe314
- IRC: ThomasMonroe TMcSquared
- In-game: ThomasMonroe TMcSquared
- Location: Wherever I am at
Re: Python as a modding language
Guys, I didn't want this to be an argument about whether or not Python is better than Lua, else I would have titled the topic differently.
What I wanted was to see how many people would have liked python as a modding language.
And furthermore, this is not a proposal to add it into the Minetest engine. I just wanted to see people's opinions on this, and at the moment, the votes are 14-14.
What I wanted was to see how many people would have liked python as a modding language.
And furthermore, this is not a proposal to add it into the Minetest engine. I just wanted to see people's opinions on this, and at the moment, the votes are 14-14.
I don't make messes, I just, er...disturb the local entropy!
Re: Python as a modding language
Realistically the dev team doesn't have time to monkey with a new scripting language and until you want to pay them a salary the most you can do is beg. That's very very fair and lets take a moment to acknowledge the absolutely fantastic work that they have been doing and are currently doing. The code and game functionality are *very* mature and very well maintained. That's all volunteer work so hats off to the contributors.
A more capable language would help out making rich game content beyond the basic place block, swat mob style games. One aspect of Mine* games is the non-linear game play: you can go over, around, through, whatever you want. But all I've seen have very limited game content beyond building. I'm trying to make a non-linear questing / job system that isn't the regular "kill 10k boars" but is focused on more abstract objectives with small micro actions. Of course this can be done in Lua. But it is soooo much harder than it needs to be.
I want to see Python added but I also realize it is a ton of work to move it into the core. Embedding the Python interpreter itself is not that bad with something like CFFI but that's only the tip of the iceberg. The minetest object has to be ported, security, memory management, testing testing and more testing. Windows, Android, Linux + flavors, BSD, Apple, all that stuff. It has to be done in a fork and matured then romanced into the main codebase.
Lua still sucks, Emacs for life, \r\n is technically the correct line end. ;)
A more capable language would help out making rich game content beyond the basic place block, swat mob style games. One aspect of Mine* games is the non-linear game play: you can go over, around, through, whatever you want. But all I've seen have very limited game content beyond building. I'm trying to make a non-linear questing / job system that isn't the regular "kill 10k boars" but is focused on more abstract objectives with small micro actions. Of course this can be done in Lua. But it is soooo much harder than it needs to be.
I want to see Python added but I also realize it is a ton of work to move it into the core. Embedding the Python interpreter itself is not that bad with something like CFFI but that's only the tip of the iceberg. The minetest object has to be ported, security, memory management, testing testing and more testing. Windows, Android, Linux + flavors, BSD, Apple, all that stuff. It has to be done in a fork and matured then romanced into the main codebase.
Lua still sucks, Emacs for life, \r\n is technically the correct line end. ;)
-
- Member
- Posts: 818
- Joined: Tue Apr 14, 2015 01:59
- GitHub: raymoo
- IRC: Hijiri
- In-game: Raymoo + Clownpiece
Re: Python as a modding language
I think Python would be more useful than Lua, with even just the standard libraries available, since it has a more useful set of data structures than just tables. I don't know though how sandboxing would work in Python.
I don't think it is really feasible to get Minetest using Python, though. And it wouldn't be my vote for language to switch to anyway.
I don't think it is really feasible to get Minetest using Python, though. And it wouldn't be my vote for language to switch to anyway.
Every time a mod API is left undocumented, a koala dies.
- Linuxdirk
- Member
- Posts: 3219
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: Python as a modding language
FTFYSkulls wrote:A more capable language modding API would help out making rich game content
- rnd
- Member
- Posts: 220
- Joined: Sun Dec 28, 2014 12:24
- GitHub: ac-minetest
- IRC: ac_minetest
- In-game: rnd
Re: Python as a modding language
Can you give example of 'more useful' data structure in python? I looked https://docs.python.org/2/tutorial/datastructures.html and see for example lists (lua has that), Sets (easily implemented using hashtable and operations like union(a,b) =u where for k,v in pairs(a) u[k] = v; for k,v in pairs(b) u[k] = v; too) ...Byakuren wrote:I think Python would be more useful than Lua, with even just the standard libraries available, since it has a more useful set of data structures than just tables.
I find lua more than capable to do pretty much anything short of intensive numeric simulations. Just intensive stuff like pathfinding or other similiar world operations should be written in c++ and included in api. Don't take my word for it, look for example http://torch.ch/
Code: Select all
Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first. It is easy to use and efficient, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.
A more capable language modding API modders would help out making rich game content
1EvCmxbzl5KDu6XAunE1K853Lq6VVOsT
-
- Member
- Posts: 818
- Joined: Tue Apr 14, 2015 01:59
- GitHub: raymoo
- IRC: Hijiri
- In-game: Raymoo + Clownpiece
Re: Python as a modding language
torch is fast because the parts that need to be fast are written in C.rnd wrote:Can you give example of 'more useful' data structure in python? I looked https://docs.python.org/2/tutorial/datastructures.html and see for example lists (lua has that), Sets (easily implemented using hashtable and operations like union(a,b) =u where for k,v in pairs(a) u[k] = v; for k,v in pairs(b) u[k] = v; too) ...Byakuren wrote:I think Python would be more useful than Lua, with even just the standard libraries available, since it has a more useful set of data structures than just tables.
I find lua more than capable to do pretty much anything short of intensive numeric simulations. Just intensive stuff like pathfinding or other similiar world operations should be written in c++ and included in api. Don't take my word for it, look for example http://torch.ch/AlsoCode: Select all
Torch is a scientific computing framework with wide support for machine learning algorithms that puts GPUs first. It is easy to use and efficient, thanks to an easy and fast scripting language, LuaJIT, and an underlying C/CUDA implementation.
A more capable language modding API modders would help out making rich game content
Every time a mod API is left undocumented, a koala dies.
- duane
- Member
- Posts: 1715
- Joined: Wed Aug 19, 2015 19:11
- GitHub: duane-r
- Location: Oklahoma City
- Contact:
Re: Python as a modding language
I have to say that I don't get this at all. I haven't yet looked at a data structure in any other language and thought, "I wish lua had that." Especially when you're talking about python or ruby, the proliferation of data types doesn't add much to the language. With C, at least they're giving you type safety and performance (at the expense of giving me a headache when I have to constantly scan kilometer-long declarations).Byakuren wrote:I think Python would be more useful than Lua, with even just the standard libraries available, since it has a more useful set of data structures than just tables. I don't know though how sandboxing would work in Python.
The worst problem I've had with using lua tables for everything (even objects) was the overhead from moving array elements, and if you think ahead, that's easy to delay until you've done all the cpu-intensive work.
Believe in people and you don't need to believe anything else.
- sorcerykid
- Member
- Posts: 1847
- Joined: Fri Aug 26, 2016 15:36
- GitHub: sorcerykid
- In-game: Nemo
- Location: Illinois, USA
Re: Python as a modding language
I've never touched Python myself, never had a need to. When it comes to object-oriented scripting, my forte is Perl and Javascript. Both languages have suited me well for many years (mostly Web-based applications, but also data analysis, and task automation as well).
I personally do not see Python as the next logical step for Minetest, since Lua is already the de facto scripting language for every subgame. I can see no overwhelming benefit to rewriting the script API to support a much smaller code-base. Just my thoughts.
I personally do not see Python as the next logical step for Minetest, since Lua is already the de facto scripting language for every subgame. I can see no overwhelming benefit to rewriting the script API to support a much smaller code-base. Just my thoughts.
- ThomasMonroe
- Member
- Posts: 286
- Joined: Tue Apr 04, 2017 16:21
- GitHub: ThomasMonroe314
- IRC: ThomasMonroe TMcSquared
- In-game: ThomasMonroe TMcSquared
- Location: Wherever I am at
Re: Python as a modding language
Guys, again, I just wanted to see how many people, if we were to hypothetically restart Minetest, would have like to see python instead of Lua.
I stress this point again: THIS IS NOT A PROPOSAL to change the modding language.
I stress this point again: THIS IS NOT A PROPOSAL to change the modding language.
I don't make messes, I just, er...disturb the local entropy!
-
- Member
- Posts: 818
- Joined: Tue Apr 14, 2015 01:59
- GitHub: raymoo
- IRC: Hijiri
- In-game: Raymoo + Clownpiece
Re: Python as a modding language
Stuff from python I would find useful in Lua are sets and their set operations, like union, differencing, etc. I can't think of anything else specifically. (Note I have not used python that much)duane wrote:I have to say that I don't get this at all. I haven't yet looked at a data structure in any other language and thought, "I wish lua had that." Especially when you're talking about python or ruby, the proliferation of data types doesn't add much to the language. With C, at least they're giving you type safety and performance (at the expense of giving me a headache when I have to constantly scan kilometer-long declarations).Byakuren wrote:I think Python would be more useful than Lua, with even just the standard libraries available, since it has a more useful set of data structures than just tables. I don't know though how sandboxing would work in Python.
The worst problem I've had with using lua tables for everything (even objects) was the overhead from moving array elements, and if you think ahead, that's easy to delay until you've done all the cpu-intensive work.
Every time a mod API is left undocumented, a koala dies.
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
There's lots of other cool stuff in Python that can really help:Byakuren wrote:Stuff from python I would find useful in Lua are sets and their set operations, like union, differencing, etc. I can't think of anything else specifically. (Note I have not used python that much)
- keyword arguments (with dict unpacking)
- full(er) regular expressions with standard syntax
- conditional expressions
- generator expressions and comprehensions
- arguably better/more complete generators and more consistent and comprehensive treatment of iterables
- useful general library functions which solve very common problems (e.g. iteration, dates/times, encoding, formatting, and parsing, math, etc.)
- (probably more, but that's off the top of my head)
Choosing to reduce the difference between languages to a simple matter of data structures or Turing completeness or whatever is too dismissive of real strengths that should not be ignored. (Heck, there's a lot about Go I don't like, but I really appreciate how it centralizes goroutines and channels, and wish more languages made concurrency and communication so elegant and clear.)
Honestly Lua is like Python lite. The one thing Lua does differently and better IMO is something few people use or have even heard about: ephemerons. (Yeah, a little ironically given my statement above, it's a data structure and associated behavior.)
Last edited by prestidigitator on Fri Jan 05, 2018 10:32, edited 1 time in total.
- Linuxdirk
- Member
- Posts: 3219
- Joined: Wed Sep 17, 2014 11:21
- In-game: Linuxdirk
- Location: Germany
- Contact:
Re: Python as a modding language
Yes, not having proper Regex leads to unnecessarily complex code and hacky workarounds to re-implement stuff over and over again. But that’s a general Lua downside and not particularly something that is pro Python as modding language.prestidigitator wrote:
- full(er) regular expressions with standard syntax
- sorcerykid
- Member
- Posts: 1847
- Joined: Fri Aug 26, 2016 15:36
- GitHub: sorcerykid
- In-game: Nemo
- Location: Illinois, USA
Re: Python as a modding language
I think this same argument could be made about Perl 5, which is one of the reasons it still stands out as one of my favourite scripting languages (the past 30 years I've programmed in countless scripting languages from Applesoft Basic, Visual Basic, QBasic, Hyperscript, OpenScript, awk, ECMAScript, etc).prestidigitator wrote: There are lots of common problems which you can often solve in Python with very little code (like a single readable line), but that you often have to "reinvent the wheel" for in Lua.
If it weren't for the memory footprint required by the interpreter, I would absolutely love for Perl 5 to be integrated with Minetest, even just for its powerful string manipulation operators, expressions, and functions (Lua's regex engine can't even come close). But the true beauty of Perl, at least in my view, is that it allows for rapid prototyping without the time spent constantly beautifying code only for a proof of concept. I can't tell you how many times I am forced to write a obfuscated routine in Lua that took only one line in Perl.
Re: Python as a modding language
You can trivially get lua to do pcre regexes as well as a wide variety of other support, through DSO's. Eg. http://rrthomas.github.io/lrexlib/ (luarocks install lrexlib-pcre2).
Lua is often miscompared (apples to oranges) with other languages and found "lacking", but people forget that essentially Python by itself also can't do anything of significance, and relies heavily on a huge variety of built-in libraries and optional components. Remember that even in python, you already have to `import os` in order to do some really basic stuff. You're just relying on the distribution of Python to provide these bits to you, but you can easily do the same with Lua, so it really isn't what sets the languages apart.
Lua is often miscompared (apples to oranges) with other languages and found "lacking", but people forget that essentially Python by itself also can't do anything of significance, and relies heavily on a huge variety of built-in libraries and optional components. Remember that even in python, you already have to `import os` in order to do some really basic stuff. You're just relying on the distribution of Python to provide these bits to you, but you can easily do the same with Lua, so it really isn't what sets the languages apart.
Re: Python as a modding language
Yeah, the difference is, you know Python as a language that comes with tons of libraries. But the interpreter itself can't do much more than the lua interpreter.
If Python would be added to Minetest you had to install all the libs yourself. You would just have the bare interpreter
If Python would be added to Minetest you had to install all the libs yourself. You would just have the bare interpreter
-
- Member
- Posts: 647
- Joined: Thu Feb 21, 2013 23:54
Re: Python as a modding language
A little off-topic, but it'd be interesting to see how the cost of crossing the C/Lua boundary with various parameters compares to serializing messages across a pipe (e.g. using CBOR). The latter could be used to build a modding system which would allow the actual mods to be built in several (potentially any) languages. Some scripting platforms like Lua and Python and Java could be done by protecting access to system calls, whereas others (e.g. straight C/C++, Go, etc.) where system calls can be made directly could be run in an OS container of some kind (e.g. Docker, or straight LXC in Linux).
Anyway, fun to ponder and maybe play with prototyping at some point. Kind of the next meta level of building an engine vs. a game. If we aren't at a level of computational performance that'll allow for that sort of thing at this point, we likely will be before long.
Anyway, fun to ponder and maybe play with prototyping at some point. Kind of the next meta level of building an engine vs. a game. If we aren't at a level of computational performance that'll allow for that sort of thing at this point, we likely will be before long.
Probably not. The idea of having "Python" without any libraries at all whatsoever is so useless that it need not even be considered. Again not an apples-to-apples comparison anyway, as all of Lua's standard libraries are included with its embedding in MT. No reason to expect it wouldn't be the same or similar with Python. At the very least you'd want a small selection of core libraries like the "built-in" functions, collections and iterators, math, regexs, etc. Whether you could (or should) strip out things like networking libraries is a reasonable question.burli wrote:If Python would be added to Minetest you had to install all the libs yourself. You would just have the bare interpreter
Who is online
Users browsing this forum: No registered users and 19 guests